Yes I'm sure it is harder when you're small, but don't use that as an excuse. Development of confidence is spiritual evolution, which you should always strive for. If the person you're dealing with is bigger and more intimidating, you just have to make up for it by being more hardcore, which will make you especially scary. In the case of a Kike, just remember that they don't like to get physical - they'd rather sue you or complain to the store manager or something, but they can't sue you just for making eye contact.