While I used to work at OpenAI, all of this is based on publicly-available information, my own ideas, general field-knowledge, or SF-gossip. - Leopold Aschenbrenner
Worth reading, specifically around "Counting the OOMs," was just updated on June 6th