The only way I've deeply experienced the Rock Band series is through the Wii releases. The first one was just a shoddy port that left out all the goodies, like online play and DLC. Granted, that last one isn't too easy of a feature to accomplish on the Wii.
Rock Band 2 is a step up, mainly because of how incredibly recognizable the setlist is without being crappy...give or take one or two songs. I still love to play that one solely when I'm in the mood for it.
However, Rock Band 3 is by far my favorite of the series. The new interface on the Wii version is incredibly handy. The setlist...decent, not as great as RB2, but it has quite a few keepers on it (ESPECIALLY Bohemian Rhapsody).
As for titles outside the main series, The Beatles: Rock Band is pretty fun for fans of the band, but it's not quite as challenging as the rest of the series.