It depends on the class.
If you are new to college, you probably need lectures, even if attendance is not taken.
I have read every textbook for every class i have taken. Some classes I never went to lectures except to turn in homework. I am not saying this to brag or anything like that, but to be realistic: i personally have been able to get As in classes (Univ. of Michigan and UW - madison) i have rarely attended lectures in most of the time, but i have never known anyone else who can do it.
How do i do it? I read textbooks, lecture notes if available (lol), derive results, work very independently, and most importantly: i work long hours. I find i know more information than many other students given how much i read. Things would be easier if i went to lecture for those classes. Not only that, but lectures provide other kinds of information not in the textbook (usually i find these things out from external sources). Why did i not go to lecture for some of those classes if it sounds like it is a lot more work? It probably was not a whole lot more work actually, but the reason i did not go to those classes was because i thought they were boring classes, because they were at 8am or 8:50am, or because i was "too busy" doing other homework lol, or because the professor was regrettably unclear even to my forgiving standards (sometimes with unreadable handwriting and muttered/scattered/insufficient explanation).
It helps to be in a good standing with your professor. If your professor wants you to be in lecture, then it is advisable that you attend. If he/she does not care so long as you perform adequately (as many physics/engineering profs are like) then no real need, but it would still help.
Lectures feed you information. It is luxurious from the standpoint of being someone who has gone through classes and only attended maybe 5 lectures throughout the entire course. Do you want to pass up that luxury? I did sometimes, but it really is a convenient thing to have someone just tell you how things work, instead of you having to read about it.
It honestly takes a knack to start being able to learn from just reading and working on your own, and it took me awhile to get it. Someone once told me i was just "better at reading textbooks" than they were, and although that is a silly remark, it is not entirely off base perhaps. For someone like me, who reads every textbook and really gets a lot out of them, i am used to the language used, things others may find vague i find entirely defined and clear, etc., i personally do not need a lot of examples to understand things (math, physics).
It is a different story in graduate school though. Last semester i had a solid state physics course that i gauge to still be at the undergrad level even though it was a "graduate" course. I never went to lecture, but i did get an A (UW - madison). Other classes like Electrodynamics i took in grad school that actually is a grad course i got an A in, but my classical mechanics course i got a B. These are all courses i did not attend lectures really. This, to me, shows that it is definitely not a good idea for me not to attend lectures in grad school all the time. (Also, in grad courses, it is a little difficult to get below a B, so this means compared to an undergrad perspective, i actually did sufficiently poorly/average in this class, although usually averages are at a BC here at UW - madison).
I study plasma physics, in grad school, the material is dense and more difficult than it was in undergrad. I need lectures and never miss my plasma physics related courses. I do not think i would be able to handle most of it on my own given the time constraints of the semester (and given my perceived capabilities), and really need a professor to lecture to me about it lol.
By the way, in lectures, they may provide guidance on things like homework.
I am a TA. I go through a lot of information in my discussion sections. My students who come to my recitations, they do well. Those who do not, they get Ds, really, the pattern is that definitive. So, at least in those isolated examples, there is a clear pattern.
If you know how to learn on your own, you will be fine. If you do not or are not sure, you should attend lectures. I recommend exercising care when making decisions like this, and do not base it off high school experience no matter how tempting.
As far as recommendation letters, again this goes back to what the professor likes their students to do. If they want you there, you should be there. If you are able to get As without attendance, and they do not mind at all, you talk to them occasionally, etc. that relationship works out fine. I have had some amazing references from profs who i got to know and just ended up liking me enough, being able to overlook the fact that i had reasons for not coming to lecture and still produce quality work, etc. So, it is not reference suicide.