Maybe I'm oversimplifying, but to me if you only care whether something hurts you/yours, then admit it and move on, there's nothing to debate. If you *do* care whether you hurt others, then listen when they say they are hurt and try not to hurt them.
@kyle and, if you only care about you and yours: why? Try caring about a stranger. It might be inconvenient, but I promise you'll feel good about it; more importantly, so will they.