I ended up finding information online on how to increase the bore with a router and rabbeting bit. It turned out great. I had to go 1" deep instead of the 1/2" I planned because the first router I used was junk and allowed the bit to slip. Really easy process and I was able to do all 4 wheels in about an hour. Total cost was less than $100 including a brand new router and now I have a few new tools in my garage. Here's a few photos
Unmodified wheel
First cut 1/4" wide (1/2" increase in bore) and 1/4" deep
Final product 1/2" wider bore and 1" deep for plenty of clearance.
