The physicality of vinyl is the big thing for me. A record in its sleeve is a work of art.
I buy CDs today for cost and ease of ripping (and because my hifi hasn't been set up for about ten years), and I note that these days nice card sleeves are the thing not nasty jewel cases. Even a simple card slip is better than a crap case.
I can't wait to get my hifi set up again, maybe with new speakers, once the divorce stinky poo is over.