If just dirty, use a q-tip with wd40 or any other mild cleaning agent. If you see rust, better to leave alone as any abrasive cleaner(wire brush, file, penknife etc.) that will clean rust will also remove the coating on the connectors that hold the fuse's. I have that problem on one of mine and it looks kaka.
In reality you can't easily get to the inner part of the fuse holders to clean them with any tool I've ever seen. I used Qtips to remove any loose crap and then some dielectric grease to minimize the effects of condensation and the water leaking design the trucks came with from GM!!
