Before I married I dated a few guys from my group of friends from university. This group included a great variety of people in terms of health, wealth, race, looks, etc. The only common point was that we were all similar age.
As I was lucky enough to be popular, I was choosy. I never wanted to date a smoker, because I find this a filthy, disgusting habit: I never wanted to date a fat man because I felt physically repelled; and I never wanted to date a black man because I find them physically unattractive.
I liked all of my friends as friends, but when it comes to physical attraction I just cannot bring myself to something more physical with a black guy.
Nobody would call me "smokeist" for not wanting to date a smoker, or call me "weightist" for rejecting a fat man; so why should I be labelled "racist" for finding black people unattractive?
Our views of what is esthetically pleasing are personal and we are entitled to them; we should no be forced to like or dislike certain characteristics. Furthermore...
Your beliefs about smokers obviously do affect your reaction to them. Likewise, it's not impossible that your beliefs about black people affect your response to their appearance. Not impossible. But not inevitable either, so let's suppose your lack of attraction is innocent. The rest of your question is: why shouldn't you comment on people's appearances in just the way you'd comment on art? Why can't we proclaim the ugliness of some person or race? Simple answer: people aren't art. There are different rules for talking about people because the impact is different. If you are tempted to openly announce your dislikes, you might want to ask yourself why.