There are around 368,000 students who took out loans whom the Business Department (BIS) does not have a current UK employment record for, or other details on earnings, according to a study by the National Audit Office.
It means that the government does not have enough information to decide whether these students should be making repayments on their loans, and if so, how much.
Reasons for avoiding payment could be because they are unemployed students living in the UK, EU students who have returned home or UK students who have moved overseas.
Under the current system, students only repay their loans when they are earning a certain salary - now set at £21,000 - and repayments are linked to their earnings.
The NAO's report concluded that in total this group has a total remaining debt of £5.3 billion.
The watchdog warns that the BIS has not done enough to establish whether borrowers with no current employment record are earning enough to repay their loans.
While many of these may not be in work, BIS, and the Student Loans Company (SLC) which helps collect payments, have carried out little analysis on how many may be working overseas, or the repayments that may have been missed, the NAO said.
It calls on the Government to improve the information they have on students who have taken out loans.
The report also says that in March, there were 157,000 people who had had no employment record for over a year. While the SLC writes to them at least once a year, it takes limited further action because it does not think this would be cost-effective.
And it notes that there are around 14,000 former students, with a total debt of £100 million, living overseas who are behind on their repayments.
While this is a small group compared to the total number of people with student loans, the SLC could take a "more targeted approach" to collecting in these areas, it says.
The NAO's study says that BIS has forecast that the total value of outstanding student loans will quadruple from £46 billion to around £200 billion by 2042 in today's prices.
At the same time, the number of borrowers due to repay is estimated to rise from three million in 2012/13 to 6.5 million in 2042.
More than a third (35%) of new loans taken out are not expected to be repaid, according to government figures, and around half of students are not expected to fully repay their debt.
Under a major overhaul of higher education funding, which saw tuition fees at English universities treble to a maximum of £9,000, student loans are now written off after 30 years.
The NAO's report says that in designing the system, the Government anticipated that a proportion of loans would not be repaid and has not set a target for the amount of money to be collected in repayments each year.
It raises concerns that BIS's forecasts of how much it will get back annually are currently around 8% higher than the amount collected and that the department cannot explain why.
NAO head Amyas Morse said: "Given the expanding size of the student loan book, BIS now needs to take a more energetic and considered approach to maximizing the value of the loan book to the taxpayer and achieving a high level of collection performance."
A BIS spokesman said: "The report demonstrates that there is an effective and efficient process resulting in high collection rates at a low cost which we believe demonstrates good value for money.
"We need to ensure that all borrowers who are earning over the relevant payment threshold are repaying their loans including those who have moved overseas after leaving their course.
"We are continually improving the collection process for borrowers and we will carefully consider the NAO's recommendations as part of this programme."
Shadow Higher Education Minister Liam Byrne said: "Labour has warned ministers time and time again that tripling fees overnight would create huge new debts that lots of students, facing a cost-of-living crisis, couldn't afford to pay back.
"In May, the Universities Minister was boasting of the government's 'text-book' reforms. Now we learn that blundering out-of-touch ministers got their sums so badly wrong that there's a £600 million hole in the department's budget.
"We need to know fast how ministers got it so wrong, and how they're going to fix it without putting the Britain's scientists, students and colleges under threat.
"Worse of all, we may be at the point where so many students loans are being written off, that the government's new student finance system is actually more expensive than the old arrangements, even though the government is asking students for three times as much money."