I agree with you that as soon as you are conscientious, careful and pay attention then you are fine and the system works for you. But the numbers don't lie and what you are saying does not apply to majority of americans: Credit card debt explodes at 2nd fastest pace on record.
Different factors have different "weight" in the final score, the weight depends on which company do you use for the score. Here are the rough weights: