pull down to refresh

Let’s Build the GPT Tokenizer: A Complete Guide to Tokenization in LLMswww.fast.ai/posts/2025-10-16-karpathy-tokenizers
187 sats \ 0 comments \ @carter 17 Oct 2025 AI
related
TOON (Token-Oriented Object Notation) — The Smarter, Lighter JSON for LLMsdev.to/abhilaksharora/toon-token-oriented-object-notation-the-smarter-lighter-json-for-llms-2f05
222 sats \ 2 comments \ @0xbitcoiner 17 Nov 2025 AI
What If We Remove Tokenization In LLMs?www.youtube.com/watch?v=G7ryb91BDG8
173 sats \ 0 comments \ @carter 2 Jul 2025 AI
Deep Dive into LLMs like ChatGPTwww.youtube.com/watch?v=7xTGNNLPyMI
630 sats \ 1 comment \ @k00b 8 Feb 2025 AI
You can ask GPT-5 to pretend it is dumber than it is
517 sats \ 0 comments \ @Tony 16 Aug 2025 AI
LRC-20 - a new approach to tokenization on Lightningx.com/akitamiabtc/status/1780253181425205586
41.9k sats \ 33 comments \ @anon 16 Apr 2024 lightning
Context Rot: How Increasing Input Tokens Impacts LLM Performanceresearch.trychroma.com/context-rot
334 sats \ 2 comments \ @Scoresby 14 Jul 2025 AI
LEAST likely next token predictorgithub.com/geeknik/least-likely-token
102 sats \ 1 comment \ @geeknik 23 Feb 2025 AI
Ranking LLMs - Sharing a Useful Spreadsheet
334 sats \ 2 comments \ @joseph_at_nostr_fan 21 Apr 2025 AI
Show HN: Wordllama – Things you can do with the token embeddings of an LLMgithub.com/dleemiller/WordLlama
231 sats \ 0 comments \ @hn 15 Sep 2024 tech
Transformer based AI will not lead us to AGI/ASI and is just a hype machine
3139 sats \ 18 comments \ @cy 2 Jul 2025 AI
Learning the Wrong Lessons: Syntactic-Domain Spurious Correlations in LLMsarxiv.org/abs/2509.21155
445 sats \ 6 comments \ @optimism 2 Dec 2025 AI
Should you use "you" in your LLM prompts?x.com/BrianRoemmele/status/1998068828295877011
827 sats \ 10 comments \ @Scoresby 9 Dec 2025 AI
GPT-5 Is More Expensive Than Claude — Here’s Why
415 sats \ 10 comments \ @Tony 10 Aug 2025 AI
LLM Agents can Autonomously Hack Websitesarxiv.org/pdf/2402.06664.pdf
474 sats \ 2 comments \ @doofus 25 Feb 2024 security
When Models Lie, We Learn: Multilingual Span-Level Hallucination Detectionarxiv.org/abs/2510.04849v1
433 sats \ 2 comments \ @optimism 19 Oct 2025 AI
Are you using a local LLM?
277 sats \ 5 comments \ @itsrealfake 5 Mar 2025 AI
MCP-Bench: Benchmarking Tool-Using LLM Agentsarxiv.org/abs/2508.20453
269 sats \ 0 comments \ @optimism 30 Aug 2025 AI
The Bitter Lesson is coming for Tokenizationlucalp.dev/bitter-lesson-tokenization-and-blt/
140 sats \ 0 comments \ @carter 25 Jun 2025 AI
Financial Statement Analysis with Large Language Modelspapers.ssrn.com/sol3/papers.cfm?abstract_id=4835311&fbclid=IwY2xjawIJNupleHRuA2FlbQIxMAABHWJxn71ESvZCS0FxEF_31oro1rwtk4rlgOst5Q4A6tuxDhxB9cgZBPizAg_aem_OAMNHiz7Vyv2bb2vt2yM0Q
222 sats \ 2 comments \ @scatman 31 Jan 2025 AI
BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMsarxiv.org/abs/2510.04721
210 sats \ 1 comment \ @jakoyoh629 25 Oct 2025 AI
Is Chain-of-Thought Reasoning of LLMs a Mirage?arxiv.org/abs/2508.01191
427 sats \ 9 comments \ @optimism 7 Aug 2025 AI