pull down to refresh

Pay-per-output? AI firms blindsided by beefed up robots.txt instructions.arstechnica.com/tech-policy/2025/09/pay-per-output-ai-firms-blindsided-by-beefed-up-robots-txt-instructions/
259 sats \ 2 comments \ @0xbitcoiner 10 Sep 2025 AI
related
I Pay You to Be a Curator or How We Seek to Consume Information
2504 sats \ 9 comments \ @softglitter2d 18 Nov 2025 econ
Publishers say no to AI scrapers, block bots at server levelwww.theregister.com/2025/12/08/publishers_say_no_ai_scrapers/
407 sats \ 3 comments \ @0xbitcoiner 9 Dec 2025 AI
Cloudflare’s new marketplace will let websites charge AI bots for scrapingtechcrunch.com/2024/09/23/cloudflares-new-marketplace-will-let-websites-charge-ai-bots-for-scraping/
320 sats \ 0 comments \ @beorange 23 Sep 2024 tech
Lawsuit: Reddit caught Perplexity “red-handed” stealing data from Google resultsarstechnica.com/tech-policy/2025/10/reddit-sues-to-block-perplexity-from-scraping-google-search-results/
176 sats \ 1 comment \ @0xbitcoiner 23 Oct 2025 Politics_And_Law
Reddit blocks Internet Archive to end sneaky AI scrapingwww.theverge.com/news/757538/reddit-internet-archive-wayback-machine-block-limit
384 sats \ 7 comments \ @Coinsreporter 12 Aug 2025 news
Perplexity is using stealth, undeclared crawlers to evade website no-crawl direcblog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/
207 sats \ 2 comments \ @0xbitcoiner 4 Aug 2025 AI
1,333,337 Sats Withheld in Controversial Writing Competition
2361 sats \ 36 comments \ @TotallyHumanWriter 4 Aug 2025 BooksAndArticles
Introducing Pay per crawl - Content owners can charge AI crawlers for accessblog.cloudflare.com/introducing-pay-per-crawl/
206 sats \ 3 comments \ @carter 1 Jul 2025 devs
At 20 years old, Reddit is defending its data and fighting AI with AIwww.cnbc.com/2025/06/28/reddit-20-fighting-ai-defending-data.html
362 sats \ 2 comments \ @Coinsreporter 28 Jun 2025 AI
Reddit has reportedly signed over its content to train AI modelsmashable.com/article/reddit-signs-ai-content-licensing-deal
413 sats \ 1 comment \ @beorange 18 Feb 2024 tech
To survive the AI age, the web needs a new business model - The Economistwww.economist.com/leaders/2025/07/17/to-survive-the-ai-age-the-web-needs-a-new-business-model
552 sats \ 9 comments \ @Scoresby 18 Jul 2025 econ
Will AI Platforms Pay Creators?
1693 sats \ 21 comments \ @kr 29 Jan 2024 tech
Crawling a billion web pages in just over 24 hours, in 2025andrewkchan.dev/posts/crawler.html
115 sats \ 1 comment \ @Scoresby 17 Jul 2025 tech
Anubis: scraper defense - "weigh the soul of incoming http requests using PoW"anubis.techaro.lol/
567 sats \ 2 comments \ @k00b 25 Mar 2025 security
A thought on JavaScript "proof of work" anti-scraper systemsutcc.utoronto.ca/~cks/space/blog/web/JavaScriptScraperObstacles
146 sats \ 0 comments \ @kepford 29 May 2025 devs
Reddit sues AI startup Anthropic for breach of contract, 'unfair competition'www.cnbc.com/2025/06/04/reddit-anthropic-lawsuit-ai.html
634 sats \ 2 comments \ @k00b 4 Jun 2025 tech
Microsoft says it’s building an app store for AI content licensingwww.theverge.com/news/873296/microsoft-publisher-content-marketplace-ai-licensing
266 sats \ 0 comments \ @lunin 5 Feb AI
Anthropic accuses China's AI labs of ripping off content - just like it didwww.theregister.com/2026/02/24/anthropic_misanthropic_chinese_ai_labs/
428 sats \ 1 comment \ @0xbitcoiner 24 Feb AI
AI Disruption: How Generative Search is Reshaping SEO and Content
140 sats \ 0 comments \ @lunin 5 Aug 2025 AI
China’s social media sites rush to abide by AI-generated content labelling lawwww.scmp.com/tech/policy/article/3323959/chinas-social-media-platforms-rush-abide-ai-generated-content-labelling-law
297 sats \ 3 comments \ @lunin 3 Sep 2025 AI
Why Can’t Robots Click The “I’m Not a Robot” Box On Websites?archive.is/K9W7B
75 sats \ 3 comments \ @Taft 6 Jun 2024 security