The former Manchester United manager had just secured his eighth Premier League title with the Reds when Abramovich approached him 10 years ago.
Ferguson told PBS' Charlie Rose: "They used an agent to approach me when Abramovich first took over the club and I said, 'No chance.'"
United have suffered their worst start to a league season in 24 years in the wake of Ferguson's retirement but the Scot insisted they are in "good hands" with successor David Moyes.
He added he harbours no regrets about his retirement and no longer gets "worked up about Manchester United's results".
"You would be throwing your money down the drain if you put any money on me coming back as a manager," Ferguson insisted. "I made my decision. The timing was perfect. There is no way back for me now. I've got a new life.
"I want to go to the Kentucky Derby and the US Masters, the Melbourne Cup. I want to visit vineyards in Tuscany and France."
In the interview broadcast on Tuesday night, Ferguson reiterated that Wayne Rooney requested to leave United a day after they clinched a 20th title last season, something which Rooney privately denies.
Ferguson's autobiography is released at the end of October and he will discuss his career during a mini-tour in Manchester, Glasgow, London, Aberdeen and Dublin.