Have you chosen you job because of the people that you work with or the people that you don't work with?
I was talking to someone the other day. She works as a freelance graphic designer. She isn't the most outgoing person in the world but she is a nice woman.
It got me thinking about your career choices.
Just because society says you have to get up at 6am, fight traffic or stand on a crowded train for two hours and then slog your guts out for 8 to 12 hours for someone who doesn't appreciate half your work, it doesn't mean that you have to follow it.
If you feel more comfortable working by yourself or working at different hours to everyone else then you should strive to build a life that works for you not necessarily anyone else.
Of course you may do better work, working by yourself.
If you are able to work by yourself then freelancing and working from home maybe the answer to for you.
Just because society says one thing doesn't mean you have to follow it's advice. Follow your heart and follow your gut and you might stumble upon your dream job.