[Bob's 66]

I'd like to pull the handle from my 420 4spd and it appears to have a pin securing it at the base....can't get it punched out. Am I missing something? Or should I get a bigger hammer!? Is there one side or the other that I should consider attacking this? Seems simple...and I'm frustrated. Thanks for any help/directions.