If it hurts your feelings that's what matters. It doesn't matter if if makes sense to him. When you love someone you don't do things that you know hurt them. This is not a court of law, you don't have to prove your right to have feelings. He needs to stop.

That said I would be offended by such a statement. I can only assume he says such things because at best he's an idiot, at worst he's a jerk.

What he said was mean, plain and simple.  Not kinda mean. Nothing about it was funny.  I personally don't think a sense of humor based on belittling or otherwise being mean to others is a good one.  I would have been hurt,  more so if he gets these little digs - oh, I mean jokes - in often.

My SO does this to me quite a bit - little put downs that come out of left field and I'm never expecting it.  Usually I don't say anything and then I go away and think about it and it makes me upset.  If you raise it with them, 100% of the time they say they are joking and you are the problem because you 'can't take a joke'.  I've decided to deal with it by saying something at the time.  Responses like 'that's not very nice', 'what did you mean by that?' or 'don't put me down'.  Then I leave it and change the subject so it doesn't escalate into a big thing.  I'm hoping if he realises he can't get away with making those types of comments by my addressing them straight away, he will stop.  I've questioned why he does it , I think he has some sort of resentment towards me.  I'm happier with my life than he is with his, so I'm just going to put it down to that.

If you don't find it funny, then it's not a  joke. Tell him to knock it off.  Sounds very passive - aggressive to me. 
 

ETA: the proper answer when someone says your "joke" hurt their feelings is "Oh no, I'm so sorry!" and then to never do it again. 

When you have to make jokes at someone else's expense, you aren't funny. You're mean. I don't care how someone tries to spin it. I mean really, would you like it if your best friend said something like this? If he can't HEAR you saying it hurts you, then spin it back on him with something "ugly but funny" (although I'm not a fan of this behavior). I have enough crap to deal with in the step world; I sure don't need insults in the form of a joke to deal with from my husband.

Micro aggressions.  They chip away at you little by little.  

Stop letting him get away with these jokes in public. When he tells a joke that you don't find funny and believe to be at your expense, with a straight face and without a chuckle say:

"I don't understand the joke. Explain it to me."

Or, more directly:

"It's not funny to make jokes at my expense. I've told you before it hurts my feelings. Stop it."

Sure, he'll get embarrassed and it will make the situation socially awkward, but he'll either have to explain his joke (which, there is no good way to explain that joke that doesn't make it sound like an insult to you) or he'll turn into an embarrassed child who has a temper tantrum.

If he throws a temper tantrum or just says "come on, it's a joke", you say:

"No, it's not just a joke to me. As your wife, I'd hope you'd have more respect for me as to not tell joles at my expense when I have repeatedly told you not to. I am telling - not asking - you again: stop it."

If he continues, you leave whatever social engagement you're at. If you're the host, tell everyone it is time to go home. If your friend/family is hosting, tell him to leave. If these jokes happen in private, walk away from him.

If that seems like a lot of work, it's because it is. It's an absurd amount of work just to get your DH to stop making derogatory jokes at your (and probably other women's) expense.

My hubby is funny, very quick witted and throws out some fabulous one liners. His sense of humor tends to run dark and someimes a bit biting. However, it is never thrown in my direction and should he say something that I don't appreciate, he always apologizes profusely. Your DH should be mindful of your feelings. Personally, I would have fired back. You should have told him he wouldn't have to worry. You'd find someone younger, with more stamina to ensure to make you limoncello. Then again, I can be a B@#$& like that. 

He will start by making fun of something someone else does, to get the laughter track going, and then he will sometimes turn it to me. The list of things to make fun of never really changes:

1. My driving. Its too slow and not aggressive enough.

2. My dancing - he describes it as arms a flailing

3. My cooking - its always delicious and I always make a mess in our tiniest kitchen.

Lately when he starts directing his "jokes" at me, I just go silent and look away or sigh and just stare at him. I have told him many many times but he doesnt think he needs to change anything. Or he does the "I just wont say anything then" thing to me.

DH has a very dry, witty sense of humor. Sometimes I don't know he's joking and I don't get the joke....I tend to the more sarcastic one liners that point out the obvious.

That said he rarely aims the joke at me. If he does and it's not funny (sometimes it is) I will tell him immediately what I think. He apologizes. And he knows what's off limits. He does the same to me. I don't think he does it to be mean at all, and maybe that's the difference? My DH doesn't have any malice in it, and neither do I.

As a child, I experienced sibling abuse for years. So I sure as heck am not going to accept poo as an adult.

When I first met DH, he tried that jockular, Alpha male bullying on me. Are you familiar with the axiom "We teach people how to treat us"? Well, I trained that poo out of him right quick. 

FIL, the career military misogynist, tried doing the same thing. It's emotionally immature behavior that's still encouraged in some male dominated fields, but I don't tolerate it. 

Shortly after we married, FIL stopped by for a visit. Instead of greeting him, chatting etc, I left the room. DH came to find out why I wasn't playing hostess. "I don't like how your father picks on me, so I'm done being around him." DH tried a little gaslighting (He's just teasing! He does that because he likes you!), but I didn't accept his blarney. So DH had a conversation with his dad, and for the next eighteen years FIL treated me with kindness and respect.

WE  decide what we'll accept and how we want to be treated, OP. Directness is your friend here. Tell your H to knock it off.

I do not have a problem with anyone telling a joke. The problem for me begins when it can be perceived as a personal dig.

EVEN the slightest hint of dig or insult.

.

That is not funny. I wont call your husband an ass, ok?

I would tell him to STOP using YOU as his punch lines. It is very hurtful.

A self depreciating joke would be you saying that "normally he is surrounded by beautiful naked women.. so we will see how it turns out this year."  when HE says that? it ceases being a joke..at least not a funny one.

Of course, it's easy to think after the fact of a good zinger.

"Gee honey.. we discussed doing it nude but we agreed there was too much risk of getting your back hair in the batch).

"Oh, Hey... I'm happy to head out the door if you miss that experience so much.. Do you think they could do some vacuuming and take out the trash?

DH has a similar sense of humour, I do as well to some extent but always come back with a zinger of my own. Like that's because he's not allowed to have the porn channel on while he makes the lemoncello anymore.

I regularly tease DH that people think he must have a lot of money when they see me with him, not true at all, but from a distance I look about 20 years younger than him.

There are times when he's gone too far and hurt me. I tell him "I know you were joking but that one hurt. please don't do it again. If I ever go too far, please tell me."

If he ever said "Fine I just won't say anything." I'd thank him for understanding and tell him I'd appreciate that.