Did a Chatbot Solve a Problem for You? \ stacker news ~AskSN

pull down to refresh

705 sats \ 10k boost \ 29 comments \ @optimism 18h AskSN

Since the word plateau is being used more often now when chatbot capabilities are mentioned, I figured that it could be a good time to start doing qualitative analysis of the tech, among stackers. I currently plan to measure this quarterly.

In the past 3 months, have you used a chatbot to solve a problem for you, in work or private life?

If you did this more than once, answer for the problem that was most important to you. ¹

Thank you for your assistance, stackers.

Bonus sats if you leave a comment with a description of which bot you tried and what it could and/or couldn't solve for you. ↩
@remindme in 3 months ↩

Yes, to satisfaction, all by itself36.4%

Yes, but I had to push/correct it39.4%

I tried, but it failed6.1%

No, I haven't tried9.1%

No, I don't use these on principle6.1%

Don't know / other3.0%

33 votes \ 7 days left

view all related items

200 sats \ 2 replies \ @unboiled 9h

In two (ie. all) cases, it was a resounding no. The bots just regurgitated what the help section already said and in both cases didn't solve my issue.

One bot eventually offered to let me speak to a human. I gave up waiting for a real person to show up.

In the other case, I was able to get an actually working answer by emailing the support address. Interestingly, I had to dig it out of some 3rd/4th level page on their site. I couldn't get the bot to give me that address, either.

21 sats \ 1 reply \ @optimism OP 9h

The on-site chatbots are truly awful and seem to still run on inferior and outdated models. I twice had one of these send me to support only to in one case be ignored, and in the other, go through what it said then in the instructions be redirected to the same bot again!

100 sats \ 0 replies \ @unboiled 8h

The on-site chatbots are truly awful and seem to still run on inferior and outdated models

Yep, I have the same impression. But many places do have humans answering if you poke the bot to get one, so I've had some success before.

For someone like me, who tries the help section first, these types of bots hardly ever tell me an option I haven't tried before.

100 sats \ 0 replies \ @astro_penguin 7h

I do not use them at all. I have a strong dislike of how they are created, what happens to people that use them, and what is happening to the internet as a result.

100 sats \ 2 replies \ @SimpleStacker 13h

How big of a problem does it have to be?

30 sats \ 1 reply \ @optimism OP 13h

Whatever was most important to you that you have tried it for.

200 sats \ 0 replies \ @SimpleStacker 12h

In that case, I'd put it this way:

Chatbots solve problems for me every day. The most common things are:
- Asking about some topic I don't know much about.
- Reminding me how to accomplish something using large libraries like matplotlib, pandas, sklearn, etc, including finding built-in functions that save me 5-10 minutes and 30-50 lines of code
- Helping me find a better word to convey a certain meaning
- Reminding me about some mathematical or econometric theorems so I don't have to go look them up
I mostly use ChatGPT and Claude. Claude is a bit better at spinning up code from scratch, I think. Chat is better at reasoning about things, including code.
The biggest problem AI solved for me was when Claude Code built a single-page applet for me from scratch that was meant to illustrate an educational point. (Basically, it runs a simulation to illustrate a point, and the user can set the parameters of the simulation.) I've never vibed anything more complicated than that.
- Overall, i'd say that for this single page site, it did a pretty good job. I had to re-prompt it to add a few features, and had to correct it once or twice, and I can't remember anymore but I think I had to go in and modify the code directly once or twice as well.
- I remember that to get it to work well, I often had to get it to reset its context memory.

200 sats \ 1 reply \ @cryotosensei 10h

I read your question several times, for you made the use of a chatbot sound rather momentous. On the other hand, I use ChatGPT so reflexively that I have downloaded the app for it haha

I have leaned on ChatGPT in various areas of my life, but you got me thinking about its most significant impact on me. I think ChatGPT has made me a more reflective writer. I typically paste my writing and ask it for a critique that covers good bad and ugly. Ngl, I kinda thrive on the glowing compliments ChatGPT give me - it’s like a morale booster haha. Anyway, having done this enough times, I have since come to realise that I have a tendency to 1) switch tone in my writing and 2) end with a weak conclusion.

I accept and acknowledge the areas of growth. Sometimes I ask for suggestions to see what ChatGPT can come up with. Other times, I brush it aside, embracing the quirks and idiosyncrasies that are uniquely mine. I think my love for writing has deepened

102 sats \ 0 replies \ @optimism OP 9h

I think that it kind of is momentous because 2 years ago (and probably even 6-9 months ago) you would have lol'd at what you're using it for today - or at least the confidence you have in it. And if it has somewhat plateau'd now - in terms of model capabilities - it means there is unresolved (and maybe unresolvable) friction still and that's worth exploring. It's cool that it's there, but does it meet up with expectations? And what does and doesn't work?

A lot of stackers are using chatbots (the usage among us seems to be really high thus far, especially if you realize that the alleged global penetration rate is about 1.5B/5.5B) but the problems we feed it are all different, and results do vary, probably through a mix of prompting effort put in, expectations and problem-space alignment. So I'd like to get a better feel for what experiences people have outside of the common ~AI commenters too.

I typically paste my writing and ask it for a critique that covers good bad and ugly.

Interesting. I generally ask "explain x" when I write about something, to make sure I that I am not completely off the mark. Like a double check. But I never say "I wrote this" - to avoid the PC. The other day I was challenging Qwen3 because it was being generic to a specific question and it actually started calling me names, lol. The training must be absurd at times.

200 sats \ 0 replies \ @TheBTCManual 9h

It doesn't happen often so I tend to remember it, recently asked a chatbot how do I capture URL parameters in an internal search query and then push the selections as custom variables into a search report for a hotel so I could give them all searches in a clean report based on their rooms, amount of guests, date range, etc and it was able to spit out a solid solution that worked first time of asking.

Was pretty stoked

200 sats \ 1 reply \ @Machiavelli23 10h

I got into a car accident a few weeks ago, no one got seriously injured just scared, I asked ChatGPT if my car was a total loss by uploading a picture of the damage and it said it was after asking a few questions. The insurance company just told me it was a total loss yesterday. I only had to upload one picture to ChatGPT and answer a 2 or 3 follow up questions.Compared that to long point less conversation with insurance agents have to upload multiple pictures and waisted time. I think that in this regard AI can be really helpful and could have people a lot of time and money. Sadly ChatGPT doesn’t work the insurance company so it hasn’t completely solved my problem but at least it gave the answers I needed to begin making predictions.

0 sats \ 0 replies \ @optimism OP 9h

Glad no one got seriously injured!

I'm personally a bit on the fence about binary / judgement questions to LLMs simply because whenever I bench a true cognitive skill "is this A or B", or "express as a % certainty" then the results are still all over the place, also with expensive models. But it's cool that it worked for you.

110 sats \ 0 replies \ @keele 12h

I don't have any chatbot service subscription so I chose "No, I haven't tried". But nearly all of my google or duckduckgo searches now display an AI overview. It's basically always wrong.

I try to keep an open mind towards new technology, but the current AI craze I don't understand. I'm a programmer and I've been doing "AI" since the 90's. I've also used ML in things like image recognition, translation and speech synthesis.

I've read multiple books on artificial intelligence such as "Behind Deep Blue" and of course "Gödel, Escher, Bach". I think Hofstadter is spot on that the paradox of self-reference is present everywhere.

I hear a lot of people say that you need to learn how to use a chatbot. It's a skill and the answers improve as your prompts get better. Well, yeah, this is self-reference in play if I've ever seen it. My problem is that the effort spent on improving the input to the chatbot might as well be spent on proper research into the topic.

For asking simple questions where I can easily spot the wrong answer it's fine. However, when I'm doing coding, I need to understand the code and code written by an LLM can't be trusted. I've also seen some of the answers Gemini returns when I've asked some medical questions and it's horrifying to be quite honest.

100 sats \ 3 replies \ @NovaRift 16h

I've used it a lot of times and as a student, AI sometimes really help me. I only use Claude, ChatGPT, and Perplexity Pro. These three work more than enough for me and my needs. ChatGPT explains better, Claude does better in technical aspects, and Perplexity is like a very smart search engine that uses GPT, Claude, and many models. I use research mode in perplexity a lot.

But the thing is, no single AI is perfect for you. You need to keep pushing them to correct it, doublecheck the response, check the sources, and reprompt them when they forget previous chats. It's a bit frustrating for me, so I just rely on them for very basic stuff and casual things like searching now. I think we're still a long way from when AI can actually help you with the things the whole hype is about. But it does help you plan things better to be honest, like groceries. It reads what a product contains, at what amount, and what it means for your health. I feel like if you use it in a way that doesn’t cross the red line (I mean, overuse), it's just like a cool tool.

42 sats \ 2 replies \ @optimism OP 15h

But the thing is, no single AI is perfect for you. You need to keep pushing them to correct it, doublecheck the response, check the sources, and reprompt them when they forget previous chats.

It really helps to switch sometimes when there's been a "model influx", like we had the past month-and-half. Try something else. The commercial products are constantly copying each other's successes to try and gain the upper hand.

I use research mode in perplexity a lot.

Example: In terms of commercial chatbot products I found that plain free Grok 4 fast currently (the last few weeks) blows Perplexity Pro out of the water on basic facts accumulation, interpretation and explanation. Like for example, Perplexity could give me some link leads in the footnotes, but Grok incorporated these much better into the answer it gave me, for the same question, and seemed to be much more aware of non-frontpage information (In this case I was asking about the Colombian debt swaps of this year)

But it does help you plan things better to be honest, like groceries.

Interesting. I haven't used it for this. How do you do that practically? Do you ask about products you were looking to buy or do you let it recommend you? Something else?

100 sats \ 1 reply \ @NovaRift 13h

Thanks, I'll try Grok. It was good, but I deleted my X account, so it went with that. I think we can now create a Grok account without having X account? I can say that Perplexity is like a hype too, and I made a wrong purchase. Still, it is what it is for general things, it's fine.

Interesting. I haven't used it for this. How do you do that practically? Do you ask about products you were looking to buy, or do you let it recommend you? Something else?

Like for example, I love stargazing, but what if I'm not sure how will be the weather of the area where I'll be there to watch the sky? Cloudy? Clear? It matters a lot because to escape light pollution, I try to go different areas, and I can ask ChatGPT or any other AI to look for weather in those areas and what time is best to view? (I prefer ChatGPT as its search function is good enough, and explanation is better and straight forward.) It can even help you plan your day if you tell your situations (like if you have an exam and need to cover topics, it will help you from where to start and how I can finish a lot of stuff in time). When I'm purchasing groceries and if I'm confused whether this food item will be healthy for me, I just give the AI picture of its labels, and it gives me an idea of what I can expect. It helps in cooking also, like I tell what stuff is in the fridge and what I can make from that stuff. There is so much you can do from AI, just see it as a tool.

25 sats \ 0 replies \ @optimism OP 13h

I think we can now create a Grok account without having X account?

It is available at grok.com where you can sign-in with email.

100 sats \ 2 replies \ @zuspotirko 16h

In my experience people who complain about having to correct chatbots often (not always) don't prompt long enough. For GPT-5 or GPT-OSS or equivalent models you don't write a Google query into a sentence. You can write 3 - 4 sentences and be surprised.

100 sats \ 0 replies \ @adlai 14h

I've actually been experimenting with minimal prompts; ideally, I'm familiar enough with the model's interpretation of the prompting structure that I can get good results without overdetermining the prompt to the point where there is no information beyond the narrow range allowed.

I think part of the defensible value proposition of these generic models is discovering your "unknown unknowns", and providing some long detailed prompt will eliminate that possibility; however, they're not as deterministic as programs that you write yourself for directly solving tasks, so they'll often fall short of the highest expectations, if you only use them as workers for solving completely-specified tasks, and even when they do perform some task perfectly, the total costs are ridiculous.

5 sats \ 0 replies \ @optimism OP 16h

This is a good point. You need to elaborate your context, so that you can get the answer you seek automatically, because you only get one answer, rather than having to click through a bunch of top search results to find the answer you need.

200 sats \ 1 reply \ @adlai 14h

Bonus sats if you leave a comment with a description of which bot you tried and what it could and/or couldn't solve for you.

I've been making heaviest use of the GPT4Telegrambot that allows selection between several different models and customisation of the SYSTEM prompt. I've been honing it for a while, and needed to make major adjustments across the switch to GPT-5.

I read lots of news channels and geopolitics discussion groups, and strongly dislike the oversimplifying false confidence given by "translate" buttons that don't give you any indication of their level of confidence, and only show you the best guess. I also believe that use of these tools should result in some IA, and not only increased dependence on AI... thus I have developed the following prompt for coping with multilingual chat groups:

Highlight evidence identifying dialect from the following sample;
and then gloss each semantic unit, including phonetic spelling,
English definition, and grammatical interpretation.

REGARDLESS OF DIALECT IDENTIFIED IN THE SAMPLE,
YOUR GENERATED RESPONSE MUST BE UNDERSTANDABLE
BY SOMEONE WHO ONLY KNOWS ENGLISH

I also fed that to GPT5 one time as the ending of a regular prompt, with the first half describing what I've been doing, no customisation of the SYSTEM prompt, to request help improving the prompt. The response overflowed the Telegram relay's message length limits, and also seemed to lean towards the "overdetermination" that I've warned against in another comment. Part of my concern is using fewer tokens, so I've currently remained with this minimal prompt.

0 sats \ 0 replies \ @optimism OP 13h

Do they have any subscription plans or is it still fully free to use?

100 sats \ 0 replies \ @OT 13h

I haven't spent much time with it. When I have tried it hasn't been able to help me much.

I'll probably continue to try as it's still pretty early and I see there's potential too.

100 sats \ 1 reply \ @adlai 14h

I think you need to narrow down the sort of problem-solving that interests you in the next survey; I've had all four of the main options on wildly different sorts of prompts, and arguably also "I don't use these on principle" because I've not been feeding them things like private chat histories, to respect my interlocutor's privacy.

5 sats \ 0 replies \ @optimism OP 14h

Good point, and noted. Thank you!

100 sats \ 0 replies \ @jakoyoh629 15h

I run with the free ChatGPT and Gemini. They sort out almost everything I need, but honestly, Google could probably do it too. It's just way quicker in the bot chats.

0 sats \ 0 replies \ @satdesigner 3h

This one yes https://chat.bitcoinsearch.xyz/

0 sats \ 0 replies \ @itsrealfake 3h

I use copilot at work. It's great for the sorts of tasks I give it.. it breaks on complicated tasks tho

I've had relatively poorer results with the open models

ive not used ChatGPT much, because I'm ethically put off by the company.

5 sats \ 0 replies \ @035736735e 17h outlawed

stackers have outlawed this. turn on wild west mode in your /settings to see outlawed content.

In the past 3 months, have you used a chatbot to solve a problem for you, in work or private life?

Footnotes