Britain's security services are "increasingly concerned" that they could be locked out from the communications of potentially dangerous suspects because of sophisticated encryption techniques, a major report has disclosed.
Police and intelligence agencies face a "significant challenge" when they are looking to monitor individuals who "pose a risk to collective security",a year-long review of surveillance practices found.
It said communications service providers (CSPs) have begun to introduce sophisticated data encryption techniques more extensively - a trend said to have been "accelerated" in the wake of the Edward Snowden revelations.
Security services are "increasingly concerned by the fact that many of the subjects of interest – including those in the highest-priority investigations– are able to use means of communication to which they no longer have access", the report said.
It added: "It is this lack of detailed intelligence available on a small number of high-priority targets that is the prime concern, rather than broader intelligence available on a large number of low-priority targets."
The report was commissioned from security think tank the Royal United Services Institute (RUSI) by then deputy prime minister Nick Clegg in the wake of disclosures by Snowden, a former US National Security Agency (NSA) employee, which prompted intense scrutiny of techniques used by American and British intelligence agencies.
It was prepared by a group of experts including Sir David Omand, a former director of GCHQ, ex-head of MI6 Sir John Scarlett and former Director General of MI5 Jonathan Evans.
Controversy has erupted in recent months over whether technology firms should be forced to hand over encryption keys to authorities when they are seeking to access the communications of suspects.
The panel concluded that police and intelligence agencies should not have "blanket access to all encrypted data", but stressed that the material should not be "beyond the reach" of law enforcement.
The report warned that a matter of "real concern" was that co-operation from technology firms has reduced.
Internet firms and CSPs told the report's authors that they are "very conscious of their corporate social responsibilities, especially in matters of terrorism and serious crime".
However, they made a "strong case" that they are "not qualified to be intelligence agencies" and "should not be assumed to be natural partners of any government in national security".
The report concluded that, despite claims that followed the Snowden revelations, there is no evidence that "the British government knowingly acts illegally in intercepting private communications" or that "the ability to collect data in bulk is used by the government to provide it with a perpetual window into the private lives of British citizens".
However, it called for the legal framework covering the interception of communications to be overhauled as it is "unclear" and has "not kept pace with developments in communications technology".
The review also found that:
::Security services face a "diffuse threat from a variety of capable and technology-literate adversaries".
::Current surveillance powers are "needed", but require a new legislative framework and oversight regime.
::The capacity to collect and analyse intercepted material in bulk should be maintained with stronger safeguards.
::The system governing warrants that allow the interception of data is "complex, incomplete and lacks legal clarity" and should be "radically overhauled". It recommends that senior judges authorise warrants requested for the prevention and detection of serious crime, while there should be judicial scrutiny of warrants relating to national security which are signed by Secretaries of State.
Chairman of the panel Michael Clarke, of RUSI, said there is a "manifest need" for new legislation.
"The government has a golden opportunity to make a fresh start by introducing legislation that provides a clear and legally sound framework within which the police and intelligence agencies can confidently operate, knowing that at all times they will be respecting our human rights," he added.
Sir John said: "Extraordinary technological change requires us to monitor rigorously the social contract, including legislation, which allows our democracy to strengthen and prosper."
Sir David said: "It is time to lift the cloud of unjustified suspicion from the digital activity of British intelligence, work that is essential to keep us safe."
Security Minister John Hayes said:"We welcome RUSI's thorough report, which is an interesting and important contribution to these considerations.
"We will consider its recommendations, alongside those produced by David Anderson and the Intelligence and Security Committee, in drawing up our legislative proposals on investigatory powers. A draft Bill will be published in the Autumn."