transfer case noise?

[bl705k]

Hey guys, I'm new to this forum and first time owner of 70 blazer. It seems you guys know your stuff so here goes. The 70 has a grinding noise coming from the transfer case or transmission especially when I let off the gas. I'm leaning toward t-case because the noise stops when I shift the case from 2 hi to neutral. The transmission is a manual four speed and the noise only appears when in fourth gear. Is it a safe bet to get the t-case rebuilt? Thanks for any reply