Here is an interesting quote from Horacio Jones:
"I like being alone. I have control over my own shit. Therefore, in order to win me over, your presence has to feel better than my solitude. You're not competing with another person, you are competing with my comfort zones."
That's an excellent point. Some people are happy just being by themselves and even though what you have planned may, in your mind be lots of fun, other people might not see it that way. If they do decline your invitation, it is not an insult to you, it is just that the other person feels more comfortable in their own company than in yours.