I suppose the obvious reason for turning any fretless players down would be tuning. Unless you are looking for a specific sound, of course (twangy plectrum with lots of attack etc...).
When a Bassist turns up and whips out his 34 incher with no staples on the fretboard and starts playing, is it possible, the band members (one of them, some of them, all of them) are hearing something in the intonation that don't like?
It's the same as with any instrument (or vocalist). If there are intonation problems going on, it's not a nice musical experience and usually ends with a polite no-no or eventually, a P45.