The contestant, who is due on stage this weekend in the next episode of the ITV1 talent show, is reported to have been claiming the money while performing on the programme.
Johnny suffers from a condition called spondylitis which leaves him with a stiff spine and he takes a daily dose of painkillers.
His last performance, a cover of The Darkness hit I Believe In A Thing Called Love, saw him lowered on to the stage before he skipped across it and around the judging panel.
A spokesman for the Department for Work and Pensions said: "We cannot comment on an individual case, but we are in contact with Mr Robinson about his situation.
"Incapacity benefit is there to support people who are unable to work and if someone's circumstances change or new evidence comes to light, they may not get the benefit any more.
"We are currently in a process of re-assessing everyone who claims incapacity benefit to see if they are fit for work."
There were similar concerns last year when it emerged that one of the finalists, Wagner, claimed incapacity benefit despite a series of energetic performances.
An X Factor spokeswoman said: "Johnny declared to us that he was receiving benefits. He is in contact with the Department for Work and Pensions due to his appearance on the show."