Of course. It's impossible to prove to a third party the reason why most failures happen.

But as you say, reputation and setup costs will limit the usefulness of that fraud. Channels have a pretty big cost to create, especially in the current system where you have in fact proved ownership of the txout.