The business minister was told by presenter Charlie Stayt that he was "deluding himself" if he thought that this week's evasive answers from the Prime Minister did not reflect badly on him.
When Stayt asked whether Cameron "got anything wrong" in the past few days, Boles replied that it was "natural human instinct" for the Prime Minister to deflect the "cacophony of attacks" on his late father.
Stayt said: "Well you must be deluding yourself if you think that Monday, Tuesday, Wednesday, Thursday the way this has come out has reflected anything other than badly on the Prime Minister.
"So on Monday this was a private matter - he wouldn't speak about it at all. We were told officially 'private matter', no comment.
"On Tuesday 'I have no shares, no offshore trust, nothing like that'. That's a very clear position.
"Wednesday, 'the Prime Minister will not benefit in the future'.
"And then finally we get the real story.
"Now you, surely, as a human being, you talked about human instinct earlier on, you can see full well what that looks like. It look like he didn't tell us the truth in the first place - didn't give us the whole picture.
"That's what it looks like and that's what it feels like."
Boles replied: "Well you can talk about the process and we are all in this game as our jobs, you and me."
Stayt interrupted: "This isn't about a game."
Boles said that he wanted to answer the question 'has the Prime Minister avoided any tax'.
Stayt pointed out that that was not the question.
He said: "If you want to answer the question, the question is 'why on day one - on Monday, why didn't the Prime Minister give us the full picture'? That is the question"
Rather than giving a 'yes or no' answer to Stayt's question, Boles used Cameron's relationship with his father as a justification for the Prime Minister's delayed response.
The Conservative minister continued to attribute Cameron's failure to reveal offshore funds to his "love" for his father.
Speaking to BBC Radio 4's Today programme shortly after, Boles said Cameron took so long to admit to owning the shares because he did not want to “intrude further on his father’s memory” who “he loves very much”.
The Prime Minister’s father, Ian Cameron, died in 2010 aged 77.
Cameron has faced intense pressure to detail his interests since the Panama Papers leaks included details of Blairmore Holdings - which used “bearer shares” to protect investors’ privacy.