Jimmy Savile thought he was "untouchable" and would regularly walk in on vulnerable Broadmoor patients while they were in the bath, it has been claimed.
The former star had "total" access to the high-security hospital in Berkshire - where he would walk in on females as they bathed, a former patient from the 1970s said.
His former personal assistant Janet Cope also claimed that Savile had free rein of Broadmoor, where she accused him of working to gain access to high-profile criminals.
She told ITV's Exposure: "Jimmy was untouchable.
"(He) was very proud of the fact that he spoke to all these very high-profile poorly criminals.
Ms Cope said that although she had never seen him abuse anyone, he was a controlling person.
She added: "I knew how he operated so I was very wary of him and I felt there was an undercurrent, there was a bad side, which not many people saw."
At Stoke Mandeville Hospital, Ms Cope said Savile held a position of real authority, adding: "He controlled everything. He controlled me. Everybody was - I'm going to use the word frightened - of him."
Savile, a former BBC TV presenter who died last year, has been accused of hundreds of allegations of child sex abuse and rape.
A former female patient told the programme Savile would walk in while female patients were having baths.
She said: "He would come in when you'd be having a bath...he would just walk in."
Speaking anonymously, she added the star once tried to touch her breast - and when she reported him she spent months in solitary confinement.
Former Conservative MP Edwina Currie told the programme she was "fairly sure" that Savile suggested himself for the role at Broadmoor in the 1980s.
She said: "He would have been seen as an extremely useful person and to have a high profile doing this would have been so much the better.
Ms Currie said that Savile unearthed a series of problems at Broadmoor and believes he planned to use information to blackmail staff there.
"We gave him every instrument that he needed in Broadmoor to prey on some extremely damaged individuals," she added.
"If Savile was alive now I would want to see him locked away for the rest of his life."
Lawyer Liz Duck told the programme she was representing nine women who claimed they were abused by Savile at Stoke Mandeville Hospital.
"Some of them have had their lives destroyed (by Savile)," Ms Duck said.
"Several of the victims did report matters to members...no action was taken."
Savile's estate has been frozen in the wake of mounting sexual abuse claims.