I finally got it out, and put the new one in. apstguy is right, I had to remove those four bolts in order to fish the wire through the column.
The bad news is that it did not fix my problem with the wipers cycling everytime I turn on the ignition. Oh well, back to the drawing board
