upper a arm question. is something wrong here?

[chewychevy67]

I am putting my chassis together and i am having a problem with my upper a arm and shaft. I cant push the shaft to the end of the stud because the rubber seal hits the edge of the ear on the frame. does this on the rear bolt of both sides. the fronts both go all the way on. I realize that shims are going to go here when its has to be allighned but could this cause me problems down the road? should i file down the ear till it fits all the way. would hate this to be a problem when i need to allighn this.