My wife said, “You really have no sense of direction, do you?”

My wife said, “You really have no sense of direction, do you?”
I said, “Where did that come from?”

The humor in this joke comes from the pun that occurs during the husbands response. He asked her "where did that come from", which colloquially means "The thing that you just said or did was a non-sequitur", but it could also mean "I do not know where that [thing] was before it arrived here". Not knowing where something came from could be a sign of having a poor sense of direction, so he is confirming his wife's statement with his response.