I wonder if I am done growing already. I am turning 16 soon but I had been 5'1" and 98-104 lbs for all my adolesence. My father is 5'8" while my mother is 4'1". My mom used to tell me that I was going to inherit my grandfather's height (or at least some of it), since he was 6'7". However, I am very disappointed at this point, and I am guessing that I won't even make it to reach my dad's height. The height predictor says that I may be around 5'6", but still I am not even that tall.
The only growth spurt I remember having was a slow one from 4th grade to 7th grade, I passed from perhaps 4'7" to 5'0" in that stage. I am not sure how tall I was during elementary school, but I always was the shortest one. In my late 5th grade I started to grow upper lip hair, not that much, and by the year I enrolled in middle school I felt already a little bit taller and my upper lip hair grew a bit. My penis also enlarged and started to grow hair around that area. When I turned 13 years old, I perhaps passed from 5'0" to 5'1" and that's where I got stuck. I started to grow a little bit of male breasts and leg hair. Some arm hair also started to spot out. By the time I got into high school I was the same as in middle school. I was expecting to have a huge growth spurt by then but it never appeared. Now I am about to go to my junior year, and I am still reminded about how short and pathetic I am in school. Everyone says that I am stuck at this height for all my life, which is surprising since I wasn't an early bloomer and I never out passed anyone in my school years. I had always been the shortest one in class.
I was born short by the way, weighting around 5.5 lbs. My family, however, say that I may still grow more, and eventually I might out reach my dad. They believe that I am a "late bloomer," but I am pretty sure I am not, unless it is possible to first grow facial and sexual characteristics then height. Now I also have hair under my chin. I can't still fully grow a moustache like my father, and my legs are full of hair now. I have small feet and small hands by the way.
I hope for a miracle to happen for me since 5'1" is a pathetic height for me. I wish I could be at least 5'7" tall. It would be great if I end up being 5'9" - 5'11" though. All my family is tall, except for my parents, and possibly I might be even smaller than my father. I hope that is not my case. I look a lot like my father by the way if that helps.
I also began with a treatment of HGH under my endocrinologist's supervision. He said my hands' growth plates are about to close.
It sounds like you were in stage 3 around the age of 13, which is fairly average, which would put you at stage 4 now. That seems to match what your doctor said, when he said your growth plates are almost closed. You can use the Tanner Stage Calculator for Boys to get an estimate. I can't offer you much else because I can't see you; your doctor, however, can see you and has more diagnostic tools available to him.
Stage 3 is when you have the largest rate of growth. Why you didn't get a large growth would be something only your doctor could tell you, but since it is two years after the fact, it would be hard for him to guess. Growth does not come after sexual characteristics show up. It comes at the same time.
HGH might give you a bit more growth, if that was what was missing. However, HGH will not stop the growth plates from closing, so there is basically a race to get in as much growth as possible in what time you have left. I don't have all the details that your doctor has, but it doesn't sound to me that you are going to get more than an inch or two more growth in if you are in stage 4.
Height comes from the combination of your parents' genes -- not your grandparents. The rule of thumb is to take the average of your parent's height, which is 4'10.5". Boys typically grow from -2 inches to + 8 inches from that average -- for you it would be a range from 4'8.5" to 5'6.5". You are currently inside that range, so though you are disappointed in your size, you are within the expected range.
I know it is not what you want to hear, but I can only offer you what I know.