The shadow Brexit secretary also used a speech at Westminster Cathedral Hall on Friday to urge the PM to give EU citizens living in the UK the right to vote.
And as Britain prepares to formally break with the bloc at 11pm tonight, he called for an end to divisions, saying: “Leave/Remain ends tonight.”
Starmer is favourite to replace Jeremy Corbyn but has been blamed by some for aiding Labour’s catastrophic election defeat by leading a push for Remain.
“I want people in this country to be able to go and work abroad in Europe. And I want people in Europe to be able to come and work here,” he said.
“I want families to be able to live together, whether that’s in Europe or here, and I want people in this country, in the United Kingdom, to be able to go and study in Europe just as they can now, and people in Europe to be able to come and study here.
“We have to make the case for freedom of movement.”
Asked to clarify if that meant he would restore that right if he was PM, Starmer said: “Yes, of course: bring back, argue for, challenge.”
Starmer did not rule out extending this right to people from other nations but said the “focus” today was on those from the EU.
The UK is to leave the EU at 11pm on Friday, but little will change until the end of 2020 when the transition period ends, bringing with it the end of freedom of movement.
EU citizens wanting to remain in the nation must apply to the settlement scheme by the end of June next year.
They will continue not to have the right to vote in Westminster elections.
That is unacceptable, according to Starmer, who criticised the government for treating them “disgracefully”.
“We need to give our EU citizens rights, not tolerance, and that starts with the right to vote,” he said.
“I challenge the government to give EU citizens the right to vote and be properly citizens of our country,” he added to applause.
“We welcome migrants – we don’t scapegoat them.”
Starmer is seen as being the leader in the race to head the party after its worst general election defeat since 1935.
He is through to the final stage – the ballot of members and supporters – along with Lisa Nandy and Rebecca Long-Bailey.
The final contender, Emily Thornberry, is still trying to win sufficient support from affiliated organisations or constituency parties.
The new leader, and the deputy, will be announced on April 4.