I think it's racist to you because you are ignorant and WANT it to sound racist. I grew up in the same environment. When I went overseas to a very mixed school and was exposed to rap music, I didn't want any part of it - not because I was raised racist or not, but because I couldn't understand it. I could not relate to anything those "artists" were "singing" about.
Again, you are being ignorant. His kid grew up in a predominantly white, middle class neighborhood. He wasn't saying that black people ARE NOT middle class, DO NOT live near white people, or even that they DON'T live in neighborhoods. Grow up and stop trying to make this into a race issue. Maybe you ARE an uneducated "lower class wigger", as you call it.
Do you know WHY sagging came to be? When the gang-bangers [or anyone else] went to jail they were not always issued their exact pant size and they were not allowed belts, as they could be used as weapons to harm themselves or others. It seems that this way of life became something to be proud of, and it invaded gangsta-rap - and has wormed it's way into hip-hop culture, hence, the rest of the free world. Expressing yourself by sagging is saying that you idolize that "thug life" which seems so glamorous to teenagers these days.
You idolize that way of life, you dress and try to look the part, but you don't live it. I'm calling poser on this one. If you can relate to the lifestyle so well, then live it or lose it - unless you really are confirming your ignorance. It's a lose-lose situation, because either way you go, you look like an idiot.