By all means, tell yourself you are a good person; the people who know you are keeping a different set of books.
The line is for the person whose entire defence, when cornered, is a summary of their own intentions. You did not mean it that way. You were only trying to help. You are, at heart, a decent sort, and you say it with the wounded dignity of someone who believes intention is the same currency as consequence. It is not, and the people around you have been quietly paying the difference for years. Notice that nobody disputes your intentions. They dispute the aftermath, which is the only part they are obliged to live in. Goodness is not a mood or a self-description offered at the end of an argument; it is what other people are left holding once you have finished being yourself at them. That phrase, a different set of books, is the knife, because it names the accounting that happens behind your back every time you explain your character while somebody else clears up the mess. It lands hardest when you are midway through insisting that you are not that sort of person, or when you ask, with genuine injury, why nobody gives you credit for trying. They do. It is simply the only thing you have ever put on the table.