cutting it off with a friend

i have a friend right now i'm probably avoiding from now on. he's hard to avoid because i work with him and he's always asking me to have lunch with him and this other guy. it's starting to piss him off a little because i've been saying no to lunch for the past week, and haven't returned any calls.

i'm still wondering if these are valid reasons for dumping a guy as a friend:

1. he is a 6'2'' 130 lb ultra skinny guy, and yet he is always talking the craziest shit like he is some kind of hardass. just because he has taken martial arts lessons in the past. we went to this one restaurant and he says really loudly with a lot of people listening 'you know sometimes you look around at all the people in the room and you just know you could kick their ass'. i mean, wtf kind of psycho just randomly says this like that?

2. he is even scarier when he is drunk. the last time we had a few drinks he turns to me and says 'i'd really like to get into a fight right now, like i'd pay you 50 bucks if you'd fight me'. again, this guys a little nuts. it's also kind of comical because he looks like a toothpick.

3. when he is dating someone, i always say stuff like 'hope your date goes well' and 'i'm glad you guys got along'. but when i try to talk to him about girls i am dating, he does his best to say something shitty about my current date:

examples:
-he found out i was dating this one girl who was really religious. he kept saying stuff like 'well i bet she is a virgin' and calling her stuff like 'the nun'. by the time i got close with this girl i had become convinced she was a virgin. and i asked her 'how's it like being a virgin?' during one of our conversations, and she gave me a weird look like - i'm not a virgin, why the fuck do you assume that i am.

-we were at a bar and this one girl kept calling me and asking me to hang out. he kept saying shit like 'you're not going over to that ugly bitches place right'. just being all negative.

-i told him i was dating this french girl and his first remark was 'aah. i always wanted to learn french'. why did he say that? fucking jackass, i'm learning the french not you.

anyways, this is another friend who is gone.