You are a second year student and still thinking about high school? Usually the college humbling process happens your first year. By second year, all of us knew we were idiots. High school was easy mode, right? Who cares about it, it has no place in discussion here. College is hard, high school never was (even with IB/AP).
Developing competency and ability takes time. You may start getting better at it at some point, keep studying, and do not give up too early on if this is what you want to do. No one starts out good at this, at least I have never met someone who has, this is what education means.
What country are you in? (The GPA measure is strange to me).
By the way, you saying you "study very hard" for exams only means *you* think you study hard. To be frank, your definition of studying hard might be flat out lazy to others (believe it). Part of college is learning how to learn, and part of it is learning how to work is figuring out how much work you really have to put forth. It can take awhile to discern out how much you really do have to work to get a high grade in most any course. The difference between a C and a B student is huge, the difference between a B and an A student is even larger, a fact that may B students underestimate.
At the moment you are doing about average compared to everyone else (as much as we can glean from only knowing limited statistics). If you want, try and get better, but getting depressed or terrified of it is not productive. Comparing absolute grades is silly as well, exams at some point start to become educational exercises, where it is not expected that students could ever get anywhere near 100%, mostly exams get too difficult to score relatively high at all. What matters as a measure is how you do compared to others mostly. That is who you are competing against. So goes the quotation from a student on a class evaluation form: "the class was very educational, everything that wasn't covered in lecture was on the exams" lol.
Sure I can relate, I used to score average in classes. In fact, I still do sometimes, it depends on the exam, how much time I have, how hard the course is, how my competency is developing, how much work I have put forth into it, etc. I also often get the high score on exams, or well above average. Sometimes I even get below average, it happens. Judging yourself based on exams should be done delicately, there are a lot of factors. Nonetheless, I started out having all sorts of trouble keeping up, understanding things, and training my intuition and problem solving ability, but I eventually got it down good enough to where I would not say I am horrible at it, and so can you. I know it is hard right now, it always will be, but you will figure things out if you stick with it hopefully. Based on you getting average scores in physics, you are at least capable enough to get a bachelors degree. How about holding off on judging your ultimate capabilities until later on in your undergraduate life? You would be surprised how much you will change in a few years in regards to adeptness, resourcefulness, understanding, and even how much better of a student you can become. Just wait and see, if a PhD is in your future, you can figure that out then. It is a little early to be thinking about quitting if you are scoring decently enough like you are. Maybe you can get better someday, who knows.
Best of luck, this is only going to get harder (unbelievably harder), so you will have to take it all in stride. Good news, you are not in this alone, you and your fellow students all have to go through this. Have fun getting an education and with the important lesson of humility. High school fooled us all, no way were we all geniuses. It should have been evident back then, but it sure was not for me and everyone I know.