I remember getting that and running around Ban’ethil Hollow in Teldrassil to scare the new players.
Light of Elune maybe, depending on how it works in Classic. Only one shot with it though.
Also, Skull of Impending Doom I think it’s called, really helps to speed up run times.
Carrot on a Stick for sure.
seal of ascension . Ony cloak. key to the grim guzzler.
Blump Family Fishing Pole
Rod that shifts you into furbolg. Don’t ever complete the quest. (Alliance)
Fishing rod/hit trinket from hinterlands. (Horde)
Nature resist items from maraudon/strat (you’ll be back for them later).
Items that shift you into gnome. (This is part of AQ quests, just don’t complete the quest chain, and the guy will give you gnome suits forever.)
Engineering items like gnomish rocket boots, teleport, and net gun. Cables you might toss eventually because they rarely ever work. After 10 failures in a row, you’ll probably feel inclined to flush it.
Some items I keep just because they have unique models. My hunter still had an arcanite reaper in the bank, long after it was obsolete. On classic, you don’t have “achievement” points. Your achievements and trophies are represented by items. My warlock had the book from Onyxia that summoned a skeleton, not because it was actually worth using for anything, but because it was a toy and trophy.
There is a google doc on /r/wowclassic
search for “Irreplaceable Quest Rewards”
There’s some sort of sticky glue you get in Westfall (and a Horde equivalent) that can be used in MC… I read this was a reliable start but for the life of me can’t remember where I read it! So if you do these quests, keep this stuff for raiding (sorry, went a little off topic there…)
Crystal of Zin-Malor it’s a beautiful thing.
There is a quest that you get that allows you to summon the Hakkar spirit in the event in Sunken Temple. Once you turn in the quest, you can never summon again.
I get this quest, but never complete it. I take the EXP hit and just kill some mobs instead. I am though, forever able to summon and start the event. 100% worth.
Dartol’s Rod of Transformation, which is not a reward but an item you use during a quest to tranform into a furbolg. Don’t finish the quest, keep the rod.
Abyss Shard, which lets a warlock summon a voidwalker without needing a soul shard. Pro tip: Use it to summon a voidwalker, park it and walk away for a free soul shard to summon a different demon.
That quest reward that looks like a ships steering wheel.
Carrot on a stick is the best I will be getting it ASAP.
This might not be in 1.12 ??
yes! i still have mine. i came across my carrot on a stick on my lock, along with all the summoning mats for the lock mount quest.
That glue will save your life. I promise, sometime on the road to 60.
like that damn alligator in the curve of the road from wetlands to ironforge.
Horde side, there’s the Alliance Captain quest real quick, the glue lets you get away with soloing it, but right away you’ll be in Silverpine, where the add’s are common over trees, its key stuff. 100% dont throw it away. 
the trinket from the warlock quest that summons a vw for free and i think it might be instant. not sure on that though.
https://classic.wowhead.com/item=20534/abyss-shard
The Horde version is called Really Sticky Glue and it’s a reward from a quest in Sen’jin Village.