In his recent Strategist contribution, David Lang nailed a critical, but sadly missing, dimension of the counter terrorism intelligence debate regarding encryption. David put it perfectly when he argued that encryption is a ‘central contributor to the health of the global economy and business competition’.

And in doing so he possibly rendered redundant Anthony Bergin’s earlier argument that the resolution of the encryption problem is about balancing civil liberties and national security in times of heightened threat. I agree with David that encryption is so essential to our economic and societal security that a compromise simply isn’t possible.

But encryption being played as part of the security versus liberty debate is likely symptomatic of three bigger problems for intelligence collection managers: an over-reliance on single source intelligence collection, a preference for technical over human source intelligence and a failure of imagination in collection management.

All of those problems find their genesis in the 1970s. For intelligence and law enforcement agencies, the late 70s were a formative period for interception methodologies. By 1973 there was a telephone in almost three quarters of Australian homes. By 1976, Australians had international subscriber dialing allowing global communications without having to access a switchboard operator: providing a new degree of privacy—a point not lost on criminals and national security targets alike.

With new interception technologies in place in the 70s, law enforcement and intelligence agencies had access to a wealth of intelligence. But importantly there was a low likelihood of those authorities being detected collecting this intelligence. In comparison to human source, undercover and surveillance operations, telephone intercepts were a low risk activity.

The threats to privacy posed by telephone interception operations were recognised in Australia by the late 70s, leading to the passage of the Telecommunications (Interception and Access) Act 1979. With the arrival of mobile phone technology in the 80s and 90s the collection of intelligence by telephone interception had become the default intelligence source for collection managers and police investigators alike. And naturally so: it provided, cheap, easy and low risk intelligence in real time.

Customers, from the halls of government to investigators in local police stations, became voracious consumers of raw intercept reporting, which allowed a fly on the wall perspective. That also led to a devaluation of human source intelligence which often has less direct access, is prone to interpretation and is less timely.

Broadly, the rise of telecommunication interception intelligence resulted in a decreased focus on human, imagery, technical and open-source intelligence capabilities. It also degraded a range of analytical tools such as traffic analysis.

For law enforcement the problems started occurring with the arrival of Blackberry and Viber. But the operational challenges of decrypting these particular platforms were symptomatic of a wider strategic reality. The Snowden leaks revealed that the US Government were using a patchwork of tools, backdoors and behind closed door relationships to counter encryption—including efforts allegedly focused on undermining the further development of the technology. Despite these efforts governments could no longer develop decryption technologies fast enough to keep pace with the technology markets. And the evolving forms of internet based communication (such as Chatsecure, Cyrptocat, jitisi) have consistently outpaced the available and emerging interception technology.

Federal Bureau of Investigation director, James Comey, has described this new operational reality in terms of interception intelligence sources ‘going dark’. Alleged criminal and terrorist targets are now using increasingly sophisticated encryption services which prevent law enforcement and police agencies from intercepting their communications. The interception intelligence sources are no longer shining a light on the covert activities of these targets.

I don’t believe that intelligence agencies want to be ‘data-holics’ as Anthony alleges. They just want access to real-time, low risk fly-on-the-wall intelligence.

Encryption is rightfully here to stay and will continue to rapidly improve. And law enforcement and intelligence agencies must continue to invest in research and development of technology solutions that are legal, affordable and practical. But this won’t fully resolve the encryption challenge.

I originally thought that this dilemma required a greater consideration of human sourced intelligence. But now I think that the solution is two-fold: greater emphasis on intelligence collection management and planning, and on shaping public and government expectations of what intelligence can do.

For 30 years, law enforcement and intelligence agencies have truncated the management of their intelligence collection through a default preference to telecommunication interception. With the degradation of this capability, those responsible for tasking the collection of intelligence must now consider alternative collection capabilities. They must also seek to employ traditional intelligence capabilities in increasingly innovative and imaginative ways. These will be challenging times for our police and intelligence officials, and there will be limited time for lamenting what was.

Intelligence customers are fickle masters. They want to know the unknowable: what the future will bring. And any surprise or negative outcome is characterised as an intelligence failure: ‘That the attacks [Paris] occurred seems less likely due to an inability to unlock encrypted communications data than due to a failure of coordination, follow-up, targeting and action’. As such, it’s time to discuss and acknowledge the uses and limits of contemporary intelligence, including the reality that it isn’t omnipresent.