Saying that the Silver Covenant is neutral is the same as saying the Sunreavers is a neutral faction, which is false.
Sure, inside Dalaran they both have to keep control of their actions due to Dalaran’s neutrality but the SC is there representing the Alliance and Sunreavers the Horde, they both keep an eye on each other but they can’t really engage on anything because, as long as they stay in Dalaran, they have to respect its neutrality.
Outside of Dalaran, however, they both show their true colors and are heavily hostile to the opposite faction. An example of this is shown just outside Dalaran on Crystalsong Forest, there are two towers with flying points, one populated by SC high elves and the other by Sunreavers blood elves and they both attack they enemy faction on sight.
You talk as the Silver Covenant is part of the Kirin Tor but in fact, it isn’t. It is an independant military force created by citizens of Dalaran to oppose the Horde, which is also well represented inside Dalaran.
This is why Dalaran’s neutrality works, they have two forces representing each side and a middle ground force (Kirin Tor) keeping the order.