I did a cam swap a few weekends ago with the engine in the truck. I went ahead and removed the whole core support, but then I was getting ready to swap the whole front clip anyway. It's a little extra work, but it gives you complete access.
If you don't remove the core support, there is a vertical brace as swb85 says that you would need to cut and bend out of the way.