Spencer, who makes the shocking revelation in his new autobiography, said an older man targeted him in Le Feeling nightclub on the island of Saint Bart's in the Caribbean.
The 25-year-old former Eton student turned reality star said: “I noticed a strange figure whose gaze seemed fixed on me. He was a skinny, much older man with bleached white hair, gold earrings and a sleeveless denim jacket.
“As my vodka went down, I started to feel unusually light-headed. I saw that the man’s cold gaze turn into a suggestive smile. It was then that I suspected he might have spiked my drink.”
Spencer, whose dad owns a hotel on the island where the alleged drugging is said to have happened, goes on to say in his book 'Confessions Of A Chelsea Boy: The Autobiography' that the man dragged him into a car.
In an extract serialised in Now, he explained: "Thankfully, the struggle attracted one passerby’s attention, who alerted my good friend Herve. When I saw him, I don’t believe I’ve ever been so happy. The relief was so great, I passed out immediately."
Spencer also spoke about taking acid with Made in Chelsea co-star Oliver Proudlock and an altercation with Colonel Gaddafi's son in extracts from his book.
This week Spencer, who is currently dating 'The Hills' star Stephanie Pratt, proudly tweeted about signing the first copies of the book: