pull down to refresh

50 sats \ 3 replies \ @optimism 2 Oct \ on: Make massive DB migrations fast II: batch, unlogged tables, defer constraints devs
If you're doing an online migration, ie you don't want downtime/contention
It may be that I haven't drank enough coffee yet so apologies if I'm being dumb but I wonder: what magic do you possess to realize both online batch migration and zero downtime?
write
preview
100 sats \ 2 replies \ @k00b OP 2 Oct
The bonus TIL is independent of a batch migration. I should’ve made that clear. It’s something I came across while trying to figure out why everything was so slow.
reply
100 sats \ 1 reply \ @optimism 2 Oct
Ah! Okay, yeah. In the past (90s, lol) I've done, once:
  1. Batch without constraints & test
  2. Double inserts into old and new (slows live down a bit but doesn't have to be a big deal)
  3. Batch with constraints for whatever gap is left
  4. Switchover with grace
  5. Disable double inserts
But tbh the double inserts thing was a lot of effort for not having a 10 minute or so downtime gap - this was on a callcenter like 911, so we thought it was worth it - but if I'd ever be asked again I'd stress the question: "are you sure you can't have downtime"?
reply
100 sats \ 0 replies \ @k00b OP 2 Oct
100%. I was telling the guys the other day that we've gotten a little anal about avoiding minor hiccups. It can eat up a bunch of energy.
reply