What is the next step for removing upper ball joints?

[dmjlambert]

This is on my 1969 CST/10 long bed pickup with stock all drum brakes. Here is where I am. I have the lower ball joint, spindle, and spring removed. I cut the heads of the rivets into quarters with an air cutoff tool, then used an air chisel to chip the heads off. I tried using a punch and hammer to punch the rivets out, but they are not budging even a little. I drilled into one of the rivets and tried punching again and did not make progress. I ground the surfaces down a little with the cutoff tool to try to find the rivet. The rivets are difficult to see, but I can kind of see the a circle in the metal on 3 of them. I also tried pounding on the ball joint from underneath and was not able to get very hard pounding done at that awkward angle. I may resort to taking the upper control arm off, but would prefer to leave it in place.