pull down to refresh
you're thinking for RAG purposes I assume
Yes, though I like querying my own local repositories and just read things. Sometimes I find that I have some really interesting but somewhat obscure papers in my repos that come back from search. But then, for example Springer wants me to commit to $35/mo for which I get to read 10 articles, and that's not close to the craziest pricing, not to mention they want to know everything about me, for no reason. All that can happen from that is bad things.
To do it right in a way Lightning nodes would be safe on it may require getting my own cabinet and iron since the ones with good iron don't budge on price.
The problem with lightning nodes is and always will be hot keys. So the bottom line question is: how feasible is it to implement something like Goog's Confidential VMs competitively? I think that this warrants some research. Technically, if full SEV coverage is attainable, it would be possible to do this on dedicated rented iron rather than needing to colocate into rackspace. Do need at least a 3rd gen EPYC (i.e. 7003), due to a hard requirement for SNP, and I think that hardware-encrypted I/O is needed, so perhaps make that an EPYC 9005. <frowny face>
Docker, LXC or full KVM VPS?
Baseline, AMD talks exclusively of KVM. But, k8s with confidential-containers is possible and widely used. I do like the idea of doing it with containers; it should be easier to build for continuity and scaling that way.
On-the-fly containerized Pubs that handle all bootstrapping would hide the complexity and "just work" from the agents standpoint
Yes so the flow could be as simple as subaccount on our secure lnd -> subaccount on a secure lnd you rent from us -> bring-your-own-secure-lnd? I think that the last option there is possible too?
Hmm, seems one would have to scrape and resell from the subscriptions if they weren't to implement LN themselves. If those are protected by IP that's a blocker, otherwise 10 articles for $35 would work to sell per-article $3.50, but the time limit is an issue. Would be an interesting sales hat to put on to convince some of those journals provide LN endpoints, I had some interest from defense sector for doing that with satellite data... long cycle though.
hot keys
That's one problem, but the issue more generally is marginal hardware.
AMD and Intel both have TEE options which can be used with containers, with the right hardware that shouldn't be too much lift to leverage. The issue is the hardware for budget VPS's is bad overall. An evenbigger risk with Lightning is the channel state storage, most of those boxes aren't redundant PSU with A/B power. Cheap supermicros or blades that have SPOF's throughout the stack. To roll your own iron correctly you need enterprise class hardware for the redundant PSU's, a A+B PDU's in the cabinet, and hyperconverged/SAN storage with redundant switches etc. There's no cost competitive way to do that with existing cloud providers, which brings me back to having to outlay for a cabinet which is yet another thing to manage.
rent from us -> bring-your-own
Doable with a migration trigger
The issue is the hardware for budget VPS's is bad overall.
Yes. Same for most dedicated server places you see advertised.
There's no cost competitive way to do that with existing cloud providers [..] another thing to manage.
I've done the AWS vs Colo calc many times for large systems over the years. The differentiator is always labor (and duration.) Though I think that the additional risk with the TEE systems are the endless streams of hardware vulns. It's been bad for both Intel and AMD these past few years.
I was looking late last year into getting hosting for a minimum Zen 5 box that I can get secured access to to set up the enclave host. This would right now cost about 20M sats over 2 years (because minimum 24mo contract); a huge commitment for a simple FOSS dev to make, perhaps even prohibitive, especially for just another development project. Those 24 cores are way too heavy too. Apple M4 is the only affordable-ish enclave-capable solution, but that doesn't help with developing things for AMD or Intel, at all.
So I'm a bit on the fence about the capabilities we need to securely run an LN node; have been for a longer time. The only solution is to not maintain much outbound liquidity on an insecure public node, and instead run a private mobile node channel for inbound. I know you hate these but in lieu of having a stable, secure home to host things, or a hosted fully secure enclave, this is all I can think of :-/
Remote hands are free or pretty cheap if you get a cabinet, you just need to provide very clear instructions of what to plug in where, unskilled labor will rack and plug. If you thought out the layout before shipping it should be done in under an hour.
Off-lease hardware on eBay, configure locally, then ship with plug and play instructions to the colo.
Older Dell R-series etc are practically given away, the risk is pretty minimal since you own the hardware and configured the OS etc. The bulk of the risk imo is in the channel states rather than the keys.
Maintenance and dealing coordinating resources for hardware failures is another job then though.
That said Lightning is inherently a hot wallet, the mental model people have of keeping large amounts in it are completely wrong... you're closer in thinking to use it as a proxy of sorts. You can have nearly unlimited inbound capacity on a VPS you barely trust and then swap it back to chain automatically at a given threshold, ensuring you never keep too much hot for more than a few seconds.
Keeping a lot of outbound hot would be where mobile becomes acceptable since you may want to manually approve spends.
There's a few colo providers that have specialized racks for mini PCs, including mac mini's and pi's. Good for individuals, but not really dense enough for scaling as a service. That gets back to the bigger risk of channel state corruption with SPOF, but a more sophisticated user can mitigate that with streaming backups... the more casual users you really need to ensure there's no SPOF in getting writes to disk.
Never heard of this till now, but it's an interesting category altogether... you're thinking for RAG purposes I assume
I've actually 3/4ths built this 2-3 times already over the years but never prioritized it through the door, motivation was always to be the basis for Pub in a SaaS'y context and less of its own thing but with the own-thing option... Actually made a deal for credits last year with one of the few good providers to build it, but they wouldn't commit to Hetzner-like pricing even if we scaled it.
To do it right in a way Lightning nodes would be safe on it may require getting my own cabinet and iron since the ones with good iron don't budge on price. Lightning actually would drive the cost down significantly because vendors can get killed on charge-backs if they don't introduce a ton of KYC friction.
Are you thinking docker hosting + volume storage + reverse proxy directives w/ SSL... LXC containers, or full KVM VPS? Thing I cooked up last year was an LXC stack but came around to leaning Docker as better for Pub specifically.
Yea only the service needs the identity, the payer comms can be ephemeralish if there's no async response needed.
All the bootstrapping is done on the Pub side currently, the VPS/Container host would be the final piece for automatic upgrades directly from wallet (or equivalent agent kit) without having to roll over static identifiers like offers.
On-the-fly containerized Pubs that handle all bootstrapping would hide the complexity and "just work" from the agents standpoint, that seems to be what the "serverless function" people seem to be scratching at, but then their MITM invoices anyway to fire up the function which defeats the purpose.
Making it an always on cloud appliance on the other hand might drive up the cost hurdle substantially. Maybe we could make Pub+LND library unto itself that runs as child processes to whatever agent daemon people build...