Office hours in college/university are the times when your professor expects to be in his/her office, and thus available if you have further questions.
It's not really intended as "homework help" and certainly isn't for "extra credit." Instead, it's when we (professors) make ourselves available to you (students). You don't need to go unless you have questions that you didn't get answered in class, but you're welcome to come whenever you have questions.
Office hours are usually printed on the cours syllabus and usually on your professor's office/lab door.
Your visit to office hours could be one-on-one, but may not be. It depends on your classmates and the topics generating questions. My office hours often have NO ONE coming by, but sometimes get five or six students at the same time, often with the same questions. Some students come by a lot, and others never come by.
Here's the important part... *IF* you're doing poorly (but let's not plan on that...) and you're *not* going to office hours, then you're not trying. So if you have trouble... GO! And don't wait until the day before the exam to let your professor know that you don't understand something from the first week of class. We use office hours to figure out if the class is understanding. If you (the class) demonstrate that you're not getting it, I'll usually go back and re-address the topic for the entire class. But I can't do that if I don't know until it's too late.
Don't stress about office hours. They're for you to use as you need them. Good luck in college!