Is it moral to use brain-enhancing drugs that have no negative consequences?

A well known neuro-pharmacologist once explained that there are no free rides with drugs-- no brain enhancing drugs that don't have serious negative consequences but if there were I can't imagine what would be morally wrong with using them. Suppose, for instance, that these positive effects followed from eating a certain type of vegetable - would it be wrong to eat the plant? I don't believe so. Now, inasmuch as we often compete on the basis of mental performance, it would be nice to think that everyone had access to this drug. Suppose, for instance, that only millionaires could afford it. Would it be wrong to take it then? Maybe so. But the rich already enjoy vast advantages in the foot race of life, so I don't know that this would be any different. It would just add to the inequalities that we already live with in our society.