Former failing headteachers have been recruited by Ofsted to inspect schools, it was reported on Tuesday.
Governors and ex-school secretaries have also taken on the job of Ofsted inspector despite never teaching a class, according to an investigation by BBC's File on 4 programme.
Ofsted's chief inspector Sir Michael Wilshaw admitted that the watchdog uses some inspectors to look at areas other than teaching, but added that if there are inspectors who have failed as heads, or never taught, then that needs to be dealt with.
It comes amid an increasing number of complaints from schools about inspections since changes to the regime were introduced in January.
At least two former heads who were forced out of schools that were declared as failing are currently working as inspectors, the programme reported.
And Baroness Perry of Southwark, who was chief inspector of schools during the 1980s, told the BBC she had concerns about the experience of some inspectors.
She said she had been told that former school secretaries and governors had been appointed to the post, rather than ex-teachers.
"They haven't actually been teachers and can't share that classroom experience with the teachers or with the schools they're inspecting," Baroness Perry said.
"I'd be very interested to know how Ofsted assures itself that all the people involved in inspections do in fact meet the best of those criteria."
The latest Ofsted figures show that almost half of schools inspected in the first three months of this year were found to be not good enough.
Of these, 34% - 658 in total - were only satisfactory, while a further 9% - 183 - were judged to be inadequate and either given a notice to improve or put in special measures.
This is a marked increase in the number of schools judged not good enough, which Ofsted put down to changes to the inspection regime.
Since January, schools previously rated "outstanding" are now not routinely inspected and those considered "good" are visited less frequently.
Some 262 schools have complained about their inspection, File on 4 reported.
Ofsted confirmed there had been a "modest increase" in complaints.
Sir Michael told the BBC that some inspectors had been used to assess areas other than teaching, but pledged to look at the issue of inspectors who are failed heads or have never taught.
"If that's happening, we need to address it," he said.
"When an inspector is in a classroom judging teaching I would expect them to know what good teaching looks like."
The National Association of Head Teachers (NAHT) recently asked schools to report their experiences of Ofsted inspections.
The union has launched a new website called School View, which it has described as an attempt to "capture the real picture" of what happens during school inspections.
The move comes following concerns from NAHT about the "variable quality" of inspection teams.
NAHT general secretary Russell Hobby said today: "We have been encouraging members to complain and there has been an uptick in the number of complaints.
"One in 12 is reasonably significant. There are 22,000 schools, but 2,000 or so are inspected, that shows disquiet about this. We are very worried about the quality of inspection teams."
Mr Hobby said that members have reported that sometimes inspectors are knowledgeable, fair and interested in the school while other are in a rush and do not know anything.
In some cases, the inspector has no experience in a type of school, for example an inspector with experience of secondary schools inspecting a primary.
Sir Michael told the BBC that the majority of responses to inspections were positive, adding that the bar has been raised on inspections.
"Every school has the right to complain and there's nothing to stop them complaining," he said.
"Remember there are 22,000 schools and the number of complaints we get are very, very few indeed."
An Ofsted spokeswoman said: "We are putting our best people in the field and last month we announced a scheme to train outstanding headteachers to undertake a number of inspections every year.
"If successful, we will extend the scheme over the coming years with a view to having outstanding headteachers as part of every inspection team."
Brian Lightman, general secretary of the Association of School and College Leaders (ASCL), said: "We have been raising concerns with Ofsted since January about the inconsistent quality and approach of inspection teams which in some cases is leading to inaccurate judgments.
"Schools accept that bar has been raised with inspection, but what they cannot tolerate is careless and inconsistent procedure, especially when the stakes are so high. According to our records, at least 8% of secondary schools inspected since January have registered complaints with Ofsted, which is double the figure from last year.
"These aren't just schools that have gone into special measures, they include schools with a whole range of inspection outcomes.
"Inspection is a critical part of the accountability system and it is essential that it is done properly and to the highest standard by appropriately qualified inspectors who have recent, relevant experience in the sector. Some inspection teams are very good but others have raised serious concerns.
"We are pleased that Sir Michael Wilshaw has recognised this, and that he is taking steps to address this, but it must be addressed as a top priority if Ofsted wants to maintain credibility with the profession and the public.
"Ofsted must make sure all inspectors are fully trained and have robust quality assurance to make sure that inspectors take a consistent approach."