Ive been using a duster with some silver polish wrapped around a credit card for years with the most excellent results. Pinch the duster behind the card, and it glides nicely over the top of the frets in circular motion, not risking any contact/damage to the fretboard at all.
I usually put lemon oil on a rosewood board first, that makes any stray polish really easy to clean off the wood.
whilst im at it i usually use the corner of the duster covered card to get down the sides of the frets into the corners to get the gunge out. I've been doing this for 10 years or more at lest once every 3 months with excellent results.
Silver polish is very mild, but If a more abrasive polish is necessary to remove deep scratches, brasso can be used.