Dawn Copley, who filled the role following Crompton's suspension on Wednesday, stood down after it emerged that her conduct was being investigated by another force.
Another candidate will now be sought, South Yorkshire's Police and Crime Commissioner Dr Alan Billings said.
Crompton was suspended after jurors found that the 96 football fans who died in the disaster 27 years ago were “unlawfully killed”.
Hillsborough families have levelled heavy criticism at South Yorkshire Police force and the Yorkshire Ambulance Service for their actions and have called for the removal of their most senior members.
Dr Billings said Copley had "offered to step back to her substantive role" due to her not wanting "any further negative publicity or criticism to be levelled at the force".
He said that Copley had declared the allegations into her conduct when she applied for the post of deputy chief constable.
Dr Billings said: “As this matter has not yet been concluded I am unable to comment publicly on the allegations and the outcome but as soon as I am able, I will do so.
"In the meantime Mrs Copley has my full support.
"However, Mrs Copley has made clear to me that she does not want any further negative publicity or criticism to be levelled at the force.
"In the interests of the force and the workforce she has therefore offered to step back to her substantive role to allow me to seek support from the College of Policing in identifying another chief officer from outside the force to act as temporary chief constable until a recruitment process can take place."
He said she would stay in post for a short amount of time to deal with pressing matters, but this would be kept as short as possible.
He added: "Events have moved very quickly over the last few days.
"I am sorry that I cannot say anything more definite at this time but we have been working closely with national bodies about these issues and hope to make a further announcement in the coming days."
This is the latest embarrassment for the force, which has been heavily criticised for their handling of the aftermath of the Hillsborough tragedy.
It emerged the secretary of the South Yorkshire branch of the National Association of Retired Police Officers (Narpo) had posted online that officers had done a "good job" despite the deaths of the 96 fans.
Rick Naylor, whose comments have since been removed, said ex-officers acted with "dignity" despite the "bile and hatred" aimed at them.
This came just days after a jury found those who died in the football stadium disaster were unlawfully killed and police errors contributed to the unsafe conditions.
In a message titled "It was a bad day" and accidentally posted on a website for retired South Yorkshire Police officers, Naylor wrote: "After all that transpired, it has not shaken my belief - I worked in a great police force with fantastic people who did extraordinary things.
"I am extremely proud to be an ex-South Yorkshire cop and I will hold my head up.
"South Yorkshire Police faced immense challenges in the 1980s - the steel strike, the miners' dispute and Hillsborough, and along the way we caught the Yorkshire Ripper.
"All these challenges tested SYP (South Yorkshire Police), and, yes, mistakes were made and we would all like to turn the clock back, but beyond these headlines the communities of South Yorkshire were served by dedicated police officers, full of good humour, courage, and selflessness - and that was you.
"You will be feeling sore, angry and disheartened and that is understandable but you did a good job - we all did!"
This came in stark contrast to a public press statement which said the force has "great sympathy for the grief" of those bereaved in the tragedy.
Reacting to that statement, Labour MP for Liverpool Walton Steve Rotheram told the Press Association: "lt's totally insensitive. For families - for them to have, at long last, to have some faith in the British judicial system, and that the police had changed, and that things were different - it feeds that 'us versus them' and we thought we'd put a line under that.
"I don't think there was bile and hatred towards police officers, I think the bile and hatred was one way and that was towards Liverpool fans and some of that was a direct consequence of the police being part of an orchestrated cover-up."
He said the South Yorkshire force needed to accept the verdict of the jurors.
He said: "It wasn't just a partial exoneration, it was a total and complete and absolute clearing of all of the charges against Liverpool fans. They came back with a decision of unlawful killing.
"This is a force that has showed no remorse whatsoever - they can't praise people who are the senior officers who caused the unlawful killing of 96 people."
Barry Devonside, whose 18-year-old son Christopher died in the tragedy, told the BBC: "They didn't do a good job. Yes, I saw police officers endeavouring to give mouth-to-mouth or CPR and those people were excellent.
"But the sad thing is they were only a few, maybe on two hands you could count them."
Naylor, a former president of the Police Superintendents' Association, declined to comment when approached by reporters at his detached house in Sheffield on Thursday.