You should give it up when you realize you are putting an enormous amount of time and effort into it, and getting back nothing or next to nothing from it. Case in point -- when I was a teenager, I wanted to play the classical guitar. I spent over a year studying music and practicing every day. I learned to read music and took a course on musical composition. I was able to play the Sor Studies and flamenco in a half-assed fashion. I also learned to play pop songs. But even though I wanted to play, I never felt that I was actually playing. It always seemed forced and artificial to me. A day came when I had to say to myself, "Son, you just don't have any talent for music, so you might as well give it up." I stopped playing guitar that day and haven't played it since. I've still got the guitar, but it just sits in its case (which is too bad -- it's a nice, hand-made Japanese classical guitar).