The semi-professional footballer made the decision after they shared a kiss during a challenge and Chloe then declared an interest in him.
Toby and Kaz had been coupled up since entering the villa last month, but became the second pairing to split, just a day after Aaron Francis dumped Sharon Gaffka.
Toby pulled Kaz for a chat around the fire pit, saying the progression of their relationship was “so slow” and said he wanted to get to know Chloe.
Toby then had a conversation with Chloe before they shared their second kiss.
Chloe said: “She just said that the way I went about it was a bit sneaky. Kaz was like ‘You could have kissed him anywhere else’ and I was like ‘Okay, well I did fancy him and I want to get to know him and I did what I did.’
“Do you know what, at the end of the day if you genuinely do think you’ve got something with Kaz then you got something with Kaz, you know what I mean?”
Toby then asked: “What if I’ve got something with you?”
Chloe said: “Yeah, I might have something with you. I think you have something with me, I’m pretty confident. Because I fancy you. What do you wanna do next?”
The pair then locked lips before the episode ended, leaving viewers with a lot of thoughts...
Love Island continues on Friday at 9pm on ITV2.