It’s a 5 minute walk from my house to the pub...

It’s a 5 minute walk from my house to the pub. It’s a 30 minute walk from the pub to my house.
The difference is staggering.

This is funny because of a pun. If someone describes something as staggering, usually that indicates that it is shocking or surprising, but staggering in this case is used as a verb - one which means "stumbling around", because personally the jokester was walking home while drunk.

People being drunk is also considered funny, so that might add to the humor of this joke.

I will admit - I chuckled when I read this.