The XM option came with a bigger radio that had a CD player (totally useless for me) and had some interior things that were purely cosmetic but cost a lot of money. It just wasn't worth it. Had XM been part of the base package, I would have just said yes and not activated it.
XM is a primary entry point for the emergency systems broadcast over radio, so it has it's uses.