Hey, @Donald Tobias! Ah — I'm sympathetic. A number of community members have expressed something similar. What you wrote here reminds me of something Warren Buffet was quoted saying, "It takes 20 years to build a reputation and five minutes to ruin it." I suppose I think of credit scores as a financial reputation. It takes time and consistency to build it back up. Like you wrote, a late payment can really affect your score.
Are you enrolled in any programs that can analyze your credit score? I know some major banks have the service (e.g. Chase, BOA, Capital One) — they typically provide your credit score and the breakdown on how to improve it.
If not, I'd highly recommend enrolling in something that can help you understand your score. On The Penny Hoarder, we often recommend our partner Credit Sesame for that sort of thing. Here's an article from The Penny Hoarder on Credit Sesame and how it can help: Credit Sesame Review: Can Your Credit Score Benefit From the Free Service?
Once you have a grasp on why your score is the way it is, you'll better be able to form your action plan to improving your score.
Keep us posted on your progress. We're rooting for you!