My best guess would be that it is an extra means of countermeasure/clutter rejection, as a cloud of chaff or surface return would not be the same as an aircraft or AShM.
It may also be to improve the missiles ability at retaining targetting of the initial target it was launched at instead of switching lock.
We know for a fact that many of the improvements to the 54C’s seeker (and warhead/fuze) was for “expanded capabilities against electronic countermeasures, high altitude targets, maneuvering targets very low altitude encounters and clustered targets.”
Spoiler
Making sure the missile can identify the appropriate target in cluttered environments such as very low altitude intercepts or areas in which large numbers of targets are clustered seems to be (in my opinion at least) one of the most logical reasons for a type of NCTR to be used.
Depending on how accurate the NCTR was as well, granted as I stated before, it was most likely incapable of some of the more advanced NCTR techniques such as JEM seen on F-15/F-16’s, its possible NCTR may have been used as an additional safety feature to prevent potential blue on blue incidents, seeing as from my understanding the F-14’s AWG-9 did not have NCTR, though the F-14D’s APG-71 should and the aircraft began development back around 1984, so it may have also been a feature meant to be paired with the APG-71 as well.
I thought about it some more and I dont think the NASA document proves theres a bug necessarily with the pre-launch gimbal limit of the AIM-54.
What is usually referred to as “lofting” is technically referred to as “trajectory shaping” in missiles, whereas the missile flies an optimized trajectory, not a simple predetermines flight angle. Particularly in the case of missiles with larger cross sectional areas to their motor and longer burn times such as the AIM-54, there is a larger advantage gained when flying at high altitude than smaller missiles with shorter burn times, as detailed by this equation:
Spoiler
Source
This is why loft angle would have such an effect on actual kinematic performance of the missile, and iirc, the AIM-54 is meant to aggressively climb once fired, particularly when fired against a very long range target.
This brings me to my reasoning as to why the NASA paper doesnt really prove anything necessarily. We have no idea in what method the AIM-54 used by NASA is launched, as the missile body was used due to the achievable speeds and altitudes it operates at, and it therefore carried test payloads, the warhead section, and likely the radar guidance section were likely removed, as I doubt NASA was firing the missile with a radar lock on a target with the missile guiding itself towards said target. Therefore, the missile was more likely used as a rocket, making manual lofting a necessity, as the missiles typical trajectory shaping profile was likely not being used.
I could be completely wrong on this point though, as I do not know the exact method in which they employed the AIM-54 for their experiments. I do think that this is the most logical reason behind the drastic difference in performance based solely on the launch angle of the missile by NASA, and also explains the discrepancy in stated max speed of the AIM-54 and those stated by NASA, as it was likely more used in a parabolic trajectory as a sort of dumb fire rocket with an experimental payload, rather than an advanced guided missile with trajectory shaping for optimizing intercept probability.
