Kim, 33, revealed on The Ellen DeGeneres Show: "He is going to a festival that he is in this weekend and it's his aunt's wedding and I have something work wise that came up and I might not be able to go.
"He is traveling with the baby himself and it's just... the anxiety. I'm trying to see what family member can go."
Ellen responded: "So he's never travelled with the baby alone? You don't trust him?"
"No, I'm trying to... It's going to be hard," Kim replied.
The protective mum also told the chat show host that she's already taught 14-month-old Nori to mind her ps and qs.
"She is so sweet and she is so polite," said Kim. "She says please and thank you. She says 'ta ta' - that's her thank you and she says 'p' for please."
"So she's not saying please and thank you at all!" teased Ellen, before quizzing Kim about whether there will be any more little Kardashian-Wests on the way soon.
When asked how many children she would like to have, Kim replied:
"Well, I just went to San Diego to give my pregnant sister [Kourtney Kardashian] a break. I took care of my niece, nephew and my daughter. To bathe them - all three - feed them, get them in their pyjamas and down for bed, I was literally like crying, it was so hard!
"I don't know how my mum did it with six! It was really really hard, so I'll go for two...
"But then again I have so many siblings and I love it, and I would want that for my own kids, so I'm going to do two and see how it is."
"Maybe you'll end up with six?" Ellen teased.
"I don't think six," Kim quickly replies, before pausing and adding. "But three... Maybe four."
More on Parentdish