PSA to Node Operators: Check coop-close-target-confs \ stacker news ~lightning

pull down to refresh

PSA to Node Operators: Check coop-close-target-confs

1465 sats \ 21 comments \ @antic 14 Dec 2023 lightning

I woke up this morning to find that one of my channels had co-op closed a channel that I had opened with them and that it cleared at a fee rate of 310 sat/vB, which cost me about 60K sats!

my lnd.conf was set to coop-close-target-confs=6 (a 1 hour target) as per default

Make sure in a high fee environment, you have double checked all of your settings for your own cost/risk profile.

They don't mention coop-close-target-confs but more good things to check here: https://www.lightningnode.info/technicals/highonchainfees

view all related items

111 sats \ 0 replies \ @C_Otto 14 Dec 2023

I agree. See my previous posts on nostr (and, I guess, some before on X and Mastodon, too):

1 sat \ 2 replies \ @antic OP 14 Dec 2023 freebie

Another thought, it would be great if there were a standard in lightning for nodes to publish their own policies for auto-closing channels. My channel appeared healthy to my own metrics, but with more liquidity on my side. I'm sure they had a policy of something like "if over 80% remote balance for x time, coop close" but given that that information is opaque to me (hidden in their own automation logic), I didn't have an opportunity to adjust the channel to avoid automated coop close...

0 sats \ 6 replies \ @ZezzebbulTheMysterious 15 Dec 2023

This happened to me today on my LND node, with the CPFP tx at 620 sat/vB. (reported as 2x overpayed). The force close cost almost 6x the value of the channel recovered. This channel was not even created by me. Someone established a 1M sat channel to me, and caused my node to burn 280k in fees on force close and HTLC timeout.

The CPFP transaction cost ~200k sat, with total fees being close ~280k. Total amount recovered in local balance was 50k.

This all seems broken in this environment, and is highly discouraging as a node operator. I have never had such issues with CLN.

0 sats \ 5 replies \ @antic OP 15 Dec 2023

Wait… doesn’t the channel opener pay for a force close. Did you manually initiate a CPFP? I’m puzzled as to how this could have come from your pocket.

0 sats \ 4 replies \ @ZezzebbulTheMysterious 15 Dec 2023

Yes -- my understanding was the channel opener paid for the close, I did not initiate the CPFP manually. There were 3 tx's, and the final one was claiming an HTLC timeout. It swept the hot wallet for the CPFP, then it swept it again to claim the HTLC timeout. I can share the tx's if you wish.

All this happened about 4 hours ago, without any action from me specifically.

I did not think this could happen either, but in my experience LND just burns sats...

0 sats \ 3 replies \ @antic OP 15 Dec 2023

Running a lightning node is certainly a way to spend Bitcoin.

What you are describing almost sounds like a watchtower penalty. I wonder if something got FUBAR on your node and it sent an older state in the force close, leading to a punishment txn… but I don’t see anything from today on this list: https://forkmonitor.info/lightning

Or maybe this is a successful attack vector being implemented in the wild… I am curious to see the txns if you don’t mind sharing. Just to be clear, you initiated the force close? Or they did?

It sounds like the force close was initiated while a pending ln txn was running, which is a little sus.

111 sats \ 2 replies \ @ZezzebbulTheMysterious 15 Dec 2023

Force close : https://mempool.space/tx/75d9034d016c697f59f7faa6fcf51b365a0fb29c0d96bbb253e05465d9b02362 Expensive CPFP (213k sat!): https://mempool.space/tx/318420de2ba3d7383074a9ecd45f0e9286d29f7295ca51680498611d8084d383 Expired HTLC claim (64k sat): https://mempool.space/tx/8a6ac4f11f9930c63d7c26cf524620098a4e92f9142b1b3564e96e8c91b68ebb

I am not sure who initiated the close. Its not reporting in Thunderhub, just 'closing'. I think when it is closed it reports remote or local close. I did not initiate any action from my side myself.

0 sats \ 1 reply \ @antic OP 15 Dec 2023

Ah, I see what happened now. So it looks like someone sent an HTLC through that timed out while one of you was offline. This is IHMO one of the most horrible node operator situations.

If one of the nodes is offline when a pending HTLC expires, instead of being canceled by both nodes cooperatively, it triggers a force close of the channel to resolve the HTLC on chain. If you were the recipient of the incoming transaction, theoretically you would have that HTLC balance as input from the source (prior to expire)—so what got taken from you on chain should be in your LN balance (just as a balance shift incoming from another channel). This is my understanding of it anyway. This hasn’t happened to me yet…

I’ve also heard of people saying that htlcs can just get stuck when nodes are both online and trigger this. Maybe if you are both tor only and your ping time is really long and tor just shat the bed…

0 sats \ 0 replies \ @ZezzebbulTheMysterious 15 Dec 2023

Thank you for taking a look! I think this would account for the HTLC expire tx, in that the sats are going to be in another channel.

I am at a loss on why it spent the whole onchain wallet on such high fees? I think the bulk of the sats were spent from just broadcasting the claim tx's. It blew away several small UTXOs, overspending to consolidate them.

Seems wise to have a minimal amount and few UTXO so high fees don't eat up large claims.

0 sats \ 2 replies \ @elysia 14 Dec 2023

That's why I like onchain. I wonder what Satoshi would have thought about the lightning network?

210 sats \ 1 reply \ @antic OP 14 Dec 2023 freebie

People often say things like, "Satoshi got so much right"--but I think he didn't think through some things, or just assumed that the software would evolve as needed to compensate for scaling issues. Whatever he thought, I'm certain he didn't envision 8 billion people owning their own UTXOs since it would take something like 40 years worth of blocks for that single UTXO claim to process (and that's even with segwit)--and then what? No on-chain system scales by itself--that's the blockchain CAP theorem: Security, Scalability, Decentralization, you get 2/3 onchain, for the 3rd you need a layer 2. Satoshi made the decision that Decentralization and Security were paramount and that Scalability would come later. Some cryptos mistakenly think that Scaling should be one of the 2 on-chain pillars and that's why they will fail, because they sacrificed on-chain Decentralization or Security.

0 sats \ 0 replies \ @elysia 14 Dec 2023

Good points, I agree, thanks.

0 sats \ 1 reply \ @OT 14 Dec 2023

I'd like to see LN channel agreement that is fixed. For example: These are the fees forever. Partners cannot pull more than 80% to either side.

Anyone know if there's any reason why this can't happen? Can it be gamed?

101 sats \ 0 replies \ @antic OP 14 Dec 2023

if both nodes run the same policy, it's all just code on both sides... if you trust the other node to uphold the standards you agree on, then it's doable, but there would need to be some additional kind of HTLC type deal if you wanted to be able to punish someone for ignoring the rules (aside from deciding to just close the channel on them). I think the best rules in lightning are things like ZBF where it's just a detectable setting and your node can be highlighted for having that configuration. There's no punishment for changing your mind aside from losing your label on node index sites for being ZBF. Positive re-enforcement through providing value.

0 sats \ 3 replies \ @kepford 14 Dec 2023

Does anyone know what this actually does?

allow-circular-route

Sounds like it allows or denys circular rebalances. But, does it allow it for the node or other people using your node to rebalance? I assume the former.

33 sats \ 1 reply \ @antic OP 14 Dec 2023 freebie

more context for the feature: https://github.com/lightningnetwork/lnd/pull/3915

allow-circular-route is disabled by default to prevent a peer from using you to move funds both out and then back into themselves. This doesn't prevent you from circular routing someone's rebalance through multiple hops, just that you wouldn't be the single source of rebalancing. This might not sound like an issue, but it can be part of a type of griefing attack where someone opens a connection to your node, then saturates your pending HTLCs with circular rebalances: https://github.com/lightningnetwork/lnd/issues/3771

IMHO, this is not something most small nodes need to worry about and you might have a valid use-case for doing this between two nodes that you own that are connected to each-other. It would be more useful if this feature were updated to be an allow-list of circular-routing friendly peers (so you could selectively enable this use-case but not allow random connectors to do this).

0 sats \ 0 replies \ @kepford 14 Dec 2023

Very helpful. Thank you!

0 sats \ 0 replies \ @kepford 14 Dec 2023

; If true, our node will allow htlc forwards that arrive and depart on the same
; channel.

But I am still unsure.

https://github.com/lightningnetwork/lnd/blob/master/sample-lnd.conf#L500C1-L501C11

0 sats \ 0 replies \ @kepford 14 Dec 2023

Thanks for this. Pretty sure 6 blocks in the default.