You probably used your email addy a few times (or even once) to sign up to something and it has since been "harvested" and is now part of multiple mass-mail lists.
Just about a month ago, I used a new email addy to sign up to a newsgroup. That is the only place I ever used it. After about a week, I started receiving "Your email could not be delivered" confirmations, with a virus attachment, back at that new address. That's how fast my new address was harvested and used to mass mail worms/viruses.
Half the spiders that you see crawling the site here are probably legit - Inktomi, Google, MSN, etc... The rest are probably harvesting email addresses. It's become a part of life and there is little we can do.
I would suggest getting a new email address for personal use only and another email address for online sign-ups. Even then, all it takes is for one of your friends to send an email to a list of friends and your address to be included in that list, and somewhere along the way that email lands on an infected system, and your security cycle is broken all over again.
All in all, life sucks.
