SimpleStacker

tech

Is DeepSeek a game changer for AI? - Computerphile

I think the ChatGPT scraping is still unverified afaik. They might've used it to build the RL training data I guess. 

In [this blog](https://epoch.ai/gradient-updates/how-has-deepseek-improved-the-transformer-architecture) that gets a lot more technical they note that the optimizations they made were obviously good targets for optimization in retrospect:

> None of these improvements seem like they were found as a result of some brute-force search through possible ideas. Instead, they look like they were carefully devised by researchers who understood how a Transformer works and how its various architectural deficiencies can be addressed.