How do I get those little lint balls off my pants?

No matter how many times I wash my sweatpants they always have these little lint balls on them! Anybody know what I am talking about? They usually occur more in the inner thigh area because thats where my legs rub up against eachother. But it is also everywhere else too. Also, using a lint roller does not help at all. Which is kind of ironic since based on its name, thats what its main purpose is. But whatever. Anyways, I feel like I should just throw them out because there is too many to pick off, but they are my favorite thing to wear. Anyone have any ideas how to get them off? Thanks