What's wrong with just saying, "No, I don't want to go." It's not anything personal, you just don't want to go. And there's nothing wrong with it. Seriously, it completely baffles me how people get so intimidated and afraid of using the word no. People need to use it more often, so much shit would be avoided if people did.