pull down to refresh

Using AI to parse more content than anyone could read.
65 sats \ 12 comments \ @PlebeiusG 26 Feb 2023 bitcoin
Here’s my idea: 1- gather telegram chat group exports, urls for newsletters/articles, etc 2- clean the data and feed into an AI model for training 3- prompt the model for relevant info I may find interesting (what are the main topics in the chat group, what do people agree/disagree on, where is there confusion, who seems to give the most useful answers… etc) 4- profit
I was using OpenAI’s API last night alongside ChatGPT and I went from blank slate to thinking this is possible, even for a pleb like me
Thoughts?
Anyone have experience? Seems like there are many companies pivoting and working to train models on their custom datasets so… why not us?
write
preview
related posts
0 sats \ 0 replies \ @01xCc 27 Feb 2023
Take the answers from nostr... It's hard to assess how to do this yet, but all the information will appear there without restrictions. It'll blow your mind...
reply
10 sats \ 1 reply \ @nullama 27 Feb 2023
The ability to summarize information is going to be key in the near future.
As a simple example, there is way more than 24 hours a day of video content uploaded to YouYube. And this type of data is only getting larger and larger.
If you want to be able to process most of it, you'll need some kind of AI to assist you.
There's also heaps of content in other languages, you can use AI to translate spoken data (audio and video) to your native language.
And many other sources of information are simply growing every day.
With AI you can get a daily summary of what might be interesting to you, and this can be very valuable.
But, as you might have recognized, big tech companies already figured this out years or decades ago, and your "feed" is basically sold to whoever wants to pay to be there.
reply
0 sats \ 0 replies \ @PlebeiusG OP 27 Feb 2023
You get it. Help me build this!
reply
10 sats \ 2 replies \ @timechain 26 Feb 2023
Definitely heard of someone recently who was getting AI to transcribe podcasts, then summarize them for him. Not sure, but I think it was this episode, lemme link you...
either way, you should listen if you're trying to level up on efficiency of uploading knowledge to your brain.
https://whatismoneypodcast.com/episodes/how-to-5x-your-reading-speed-and-retention-with-emerson-spartz-wim276
How to 5x Your Reading Speed and Retention with Emerson Spartz (WiM276) Founder of Nonlinear.org Emerson Spartz joins me for an in-depth conversation on how to become much more efficient at the reading process, the 80/20 rule, and finding your unique niche that you can be the best at.
reply
0 sats \ 1 reply \ @PlebeiusG OP 27 Feb 2023
I'm trying to 1,000x my ability to upload knowledge. Why can't an AI scour hundreds of sources and relate back to me what I'd ask it to focus on? Let's say I want to keep up-to-date with topics related to bitcoin mining... just let loose the robots and have them report back what they find 😋
reply
5 sats \ 0 replies \ @Btckittykat 27 Feb 2023
If it’s just search, might be able to train a large BERT model which should be way cheaper than a LLM. Would need to plot out how much data you’re getting each time period (like daily) and then see how much it would cost to train BERT on that
reply
0 sats \ 3 replies \ @PlebeiusG OP 26 Feb 2023
In addition, this would really show its value if you were able to feed in MANY newsletters, articles and chat group logs… hundreds. No human could parse this amount of data… but if a single human uses this tool they’d basically have an executive assistant that had a bead on current chatter in a given industry.
… 🤷‍♂️
reply
10 sats \ 2 replies \ @nullama 27 Feb 2023
That's what the social media "feed" is.
Your attention is incredibly valuable, and is being sold to the largest bidders these days.
reply
0 sats \ 1 reply \ @PlebeiusG OP 27 Feb 2023
Let’s build it open-source. So that how it works is understandable and leveraged by the pleb.
reply
11 sats \ 0 replies \ @Btckittykat 27 Feb 2023
You might find Mycroft and Almond interesting, two open source digital assistant projects
reply
8 sats \ 0 replies \ @Btckittykat 27 Feb 2023
You’ll have costs to store the data and constantly training the models. Chatgpt costs millions. Also LLMs and NN in general have an issue with catastrophic forgetting so if you try to train on new data it will overwrite old data it’s learned on, meaning you have to train it on all the data at once, which means bigger and bigger training sets.
I don’t work on these myself but this my high level understanding.
Still, depends on what you want service you want to offer (a chatgpt clone might be too costly but maybe another service could be cheaper)
reply
0 sats \ 0 replies \ @01xCc 27 Feb 2023
1.bitcoin can answer you here! and if you don't get a decent answer, you'll have to ask google . Listen... what if we make a search engine based on that??? people ask and others answer??? One request, one satoshi... one answer one satoshi... number of responses you choose 1-10-100... ...and then you pay for it yourself! 2.to make a decentralized AI based on bitcoin, any request will be paid, as well as the answer. All participants will store data to interpret the AI's response. A kind of skynet, but controlled by humans... The shutdown will happen as the memory fails. It is realistic to do this. Bitcoin = instruction...
reply