Not all download sites are peer to peer sharing sites. Peer to peer file sharing is the locker room scenario we talked about earlier. Peer to peer, or P2P, sites are just conduits for users to upload and download files. Even if the site itself isn't malicious, you're trusting hundreds or even thousands of strangers to only share what they say they're sharing and nothing more.
There are more reputable peer to peer download sites out there, like RapidShare and MediaFire. Just visiting these sites shouldn't put your computer at risk, but a lot of the material on there is illegal, like pirated TV shows, movies, and software. There's nothing stopping users from sharing infected files, sometimes without even knowing it. If one user downloads a pirated episode of "Weeds" that's infected with malware, he can then share that file on a peer to peer site without realizing that he's spreading malware around.
If you're looking to download music from your favorite band, you might want to check out their official website before you do any illegal downloading. Services like Bandcamp allow bands to offer their music to fans for free, a set price or using a "pay what you want" model [source: Bandcamp]. The Bandcamp site itself isn't a file-sharing website, though. It's just a tool that bands can use to sell their music or offer it to fans for free.