yes in the abolute endgame, your BiS will not care about rares, but to get to that point you probably will use plenty of rares until you find that 3 good affix legendary.
nothing wrong with items not being useful all game all the time, i mean why not whine about blues and whites too while you’re at it? those are already useless.