You wouldn't be golden (from a vegan perspective), but you'd only have the animals you ate on your ledger.

This is one reason why I do think carnivore is often a more ethical choice for people. I've noticed that those who go carnivore tend to also avoid factory farmed meats, which are by far the worst from a moral standpoint.