I've always hated Sarah Silverman. I like goofy "nerd girls", but she's not good looking, her voice annoys me, her standup jokes do nothing but make me angry or feel sorry for her, and her political viewpoints [at least in the past] are ignorant.
Voice, looks, personality - those are the things that I think you can rate women on TV by - I like a girl that has two of those, or a good balance on a scale between them, but when you are lacking in one area, you need to make up in another. That is where she fails. If you like brown hair and eyes combo, at least you can mute Fran Drescher.