The short hose that goes between the steel vacuum line and the modulator fails on a regular basis too. That happens even more on rigs with leaks like my daily had as seen in this photo.
Make sure that hose is in good shape and connected tightly on both ends. The diaphragm in the modulator may have failed too and you would usually have fluid leaking out of the vacuum tube on the modulator if that happens.
That's a smaller size aftermarket modulator, Most of the original ones were the larger size like this one that you can see on the right side of the extension housing on the original trans from the truck.
I had a governor fail on my trans once but I don't think that the truck went at all that time let alone shift manually.
If you aren't using the second leg of the vacuum T off the manifold you can remove that T and put in a 90 with the correct size tube to connect to the modulator line and get a bit more clearance.