The circumstances of the arrest would suggest the 27-year-old suspect may have been under some form of surveillance prior to his arrest.
The man has been named by media as Khalid Mohammed Omar Ali, who was born abroad but grew up in Tottenham.
The Daily Telegraph reported that it is understood the suspect’s family had become concerned about his behaviour and reported him to the authorities several weeks ago.
The bearded man, described as “very calm” by a witness, was carrying a rucksack containing at least three knives when he was detained yards from the scene of last month’s attack by Khalid Masood.
The Metropolitan Police said he was arrested on Thursday “following a stop and search as part of an ongoing operation”, and was held at approximately 2.22pm by armed officers from the specialist firearms command.
In a separate incident a woman was shot and four people arrested as part of an ongoing counter-terrorism operation in London and Kent overnight.
Speaking of the Whitehall incident, Scotland Yard’s statement said: “The man, aged in his late 20s, was arrested on suspicion of possession of an offensive weapon and on suspicion of the commission, preparation and instigation of acts of terrorism.
“Knives have been recovered from him.
“He is being detained under the Terrorism Act and is in custody in a south London police station.
“Detectives from the counter terrorism command are continuing their investigation, and as a result of this arrest there is no immediate known threat.”
Dressed in dark clothing, the suspect was wrestled to the ground on a traffic island at the Parliament Square end of Whitehall, close to the Foreign Office and the entrance to Downing Street.
Handcuffed, with his arms behind his back, one of his hands appeared to be wrapped up in a bandage as four officers stood next to him.
Held near a red telephone box, the suspect had his black hood up as he was led away.
Images from the scene showed three knives on the ground close to a brown rucksack, and an officer in forensic overalls could be seen taking photographs of the knives.
The suspect was detained at the scene for several minutes within view of journalists and passers-by before he was eventually taken away in a police vehicle.
Westminster has been on a state of heightened alert since Masood launched his attack on March 22, killing five people, including Pc Keith Palmer.
Masood drove at pedestrians before stabbing Pc Palmer as he guarded the cobbled forecourt of the Palace of Westminster.
He was shot dead by police.
Prime Minister Theresa May was away from Number 10 on a campaign visit at the time of the incident on Thursday, which did not disrupt work in Downing Street, said a spokeswoman.
“This is an ongoing police operation and we are monitoring it,” said the Number 10 spokeswoman.
“The Prime Minister has been informed by officials and she will be updated, as usual for incidents like this.”
A French tourist, who was in New York on 9/11 and was visiting London with her daughter, saw the arrest happen.
The witness, who declined to be named, said: “I was here half an hour ago. We just saw the man arrested and that’s all.
“He was very calm. I think they got him on the ground, then they stood him up against the wall.
“There were lots of policemen. The road was blocked.
“We didn’t stay here, we just moved because I was a bit afraid because of my daughter.
“We saw the helicopter was here so we came back to see what was going on.
“I was in New York on September 11, so I am a bit scared of these things and being in the wrong place at the wrong moment.”
Another witness told Sky News the suspect seemed “relaxed” and said he looked like a “typical guy”.