I am pretty much on the other side of this "argument." Especially if you buy Nikon refurbs during a 10% off sale, the prices can be excellent. And it seems that lenses that are packaged as refurb have all been checked out and are definitely "within specs." I disagree with the assumption that refurb lenses have had "something wrong with them." Lenses get returned by buyers and then cannot be resold as brand new (for example). With one exception (discussed below), all of the refurbs I have bought from Nikon have appeared to be brand-new, shiny and unmarked. As for warranty, in my experience, usually if there is something warranty-related wrong with a camera lens you will find out pretty quickly. To make a liar of myself here, I recently had trouble with the autofocus mechanism on a Tamron 150-600mm zoom (G 2) that occurred within a year of the expiration of the six-year warranty (I bought it in 2016). So yes, length of warranty is valid consideration.
I purchased my 600mm f4 VR zoom as a refurb from Roberts. It was definitely not brand new, as it had some scuffs on it. But I felt good inasmuch as I knew it had been checked over and adjusted by Nikon to be sure it functioned as well as a new lens. The price was fair, quite a bit less than a brand new lens and I have had no trouble with it in the five years since I purchased it.