RBS has agreed to pay the UK's Financial Services Authority £87.5 million, the United States Commodity Futures Trading Commission 325 million US dollars (£208 million) and the US Department of Justice 150 million dollars (£95.8 million) for Libor fixing.
RBS chairman Sir Philip Hampton said it was a "sad day for RBS", but vowed to "put right the mistakes of the past".
RBS said 21 staff were involved in attempting to manipulate interbank lending rates - specifically Japanese Yen and Swiss Franc Libor submissions - from October 2006 to as recently as November 2010.
All 21 have left or been subject to disciplinary action and two managers with supervisory responsibilities have stepped down.
Six staff have been dismissed, including two managers, while six have been severely disciplined or are going through a disciplinary process.
Another eight left the organisation before disciplinary action could be taken and one was dismissed for misconduct not related to these findings, added RBS.
All staff that have left the bank as a result of the investigation received no bonus for 2012 and saw full claw-back of any outstanding past awards.
Hourican leaves with 12 months pay worth £775,000, but will forfeit £9 million in bonuses for last year and clawed-back previous awards.
In a statement to the House of Commons, Financial Secretary to the Treasury Greg Clark said the manipulation of Libor was "motivated by greed" and the findings against RBS were "grave".
It was "right" for Hourican to leave his post and forfeit bonuses, said the minister.
Clark told the Commons: "This is another day of shame for Britain's banks, and it is vital that we recognise it as such, not because Britain stands alone in this and similar scandals - which, as we know, is far from being the case - but because Britain must stand out in the way we put things right.
"So let there be no excuses. Instead, let us have enduring, fundamental reform and let us have justice too.
"Any organisation or individuals found guilty of a crime must take full responsibility and should be punished by law, while the ordinary taxpayer must not and will not pay the price of their misdeeds.
"If in the process we hold our financial sector to higher standards than elsewhere in the world, that is nothing to shrink away from, indeed it is something we must not only welcome but actively pursue.
"That is why we have got in place a vastly stronger system of regulation so misconduct can be prevented, not just punished, and that is why we look forward to further recommendations that will be made by the parliamentary commission.
"'My word is my bond' is the motto on which the City was built. We must rebuild that bastion of confidence here in Britain, the best place in the world to do business, but the worst place to abuse the trust on which free enterprise depends."
The settlement sees RBS pay £87.5 million to the UK's Financial Services Authority, 325 million US dollars (£207.7 million) to the United States Commodity Futures Trading Commission (CFTC) and 150 million dollars (£95.8 million) to the US Department of Justice (DoJ).
The FSA said bankers at RBS colluded with other banks to try to manipulate Libor, with one trader using false "wash" trades to bribe a broker firm.
After trawling through emails and phone conversations, the City watchdog found at least 219 requests for inappropriate Libor submissions, and countless requests made verbally.
In what will come as a blow to RBS boss Stephen Hester, the CFTC said Libor misconduct was continuing in late 2010 even after the investigation had begun.
RBS also failed to enforce any "meaningful" controls surrounding Libor submission until June 2011, according to the CFTC.
"During this time, RBS was experiencing significant growth on its yen and Swiss franc trading desks, generating revenues for RBS that were multiplying over the years," said the CFTC.
The DoJ said it was holding RBS accountable for a "stunning abuse of trust".
DoJ assistant attorney General Breuer said: "These are extraordinary results, and our investigation is far from finished. Our message is clear: no financial institution is above the law."
RBS is one of about 20 banks which are being investigated over involvement in manipulating the rate, which governs the price of more than 500 trillion US dollars-worth of loans and transactions around the world, including household mortgages.
Its fine has dwarfed the £290 million settlement agreed by Barclays last year over its involvement.
Swiss bank UBS has so far been hit with the biggest penalty, agreeing a near £1 billion settlement with regulators in December.
One of the former UBS traders who is alleged to have been involved in Libor fixing - Tom Hayes - is accused by the DoJ of colluding with counterparts at RBS.
Ex-UBS traders Hayes and Roger Darin were charged with conspiracy to manipulate the interbank lending rate following the UBS investigation. Hayes has also been charged with wire fraud and an antitrust violation.
Stephen Hester, chief executive at RBS, said: "Libor manipulation is an extreme example of a selfish and self-serving culture that took hold in parts of the banking industry during the financial boom.
"Today's announcement is not the first and will not be the last reminder of the scale of the changes that need to be made. But our determination to clean up RBS for all is undiminished."
RBS stressed that Hourican, who will leave after handing over his responsibilities, had no involvement in or knowledge of the Libor misconduct.
But the bank said both he and the board felt it was right in view of the "management issues identified in relation to this settlement and the impact on the group's reputation".
The bank's markets division headed by Hourican, which has already been dramatically reduced in size, will shrink further to focus on core corporate customers, according to RBS.
Suggested For You
SUBSCRIBE AND FOLLOW
Get top stories and blog posts emailed to me each day. Newsletters may offer personalized content or advertisements.Learn more