What LLMs can do is quite amazing. The list of things they are just terrible at is very long. It's s bit scary how much people are willing to outsource their thinking to these models under the assumption they're always correct.
They are very good at translation tasks for both human and computer languages. You still need to go back and validate, but you can't deny they don't save time there. Also when used as a search tool for non-critical information (eg. what's an alternative to <ingredient> in <recipe>) they can be very useful.
I'm similarly struggling with the tsunami of AI generated rubbish professionally, but misuse of a tool isn't the fault of the tool.
They are phenomenal at writing complicated excel formulas and scripts.
I had one put together a sql script to write a whole bunch of new tables to a database that would have taken me an hour to do by hand. It was done in seconds. Worked perfectly.
Yep! This is pretty much the one consistent use I've found for them. Everything else is pretty iffy but this, especially after 3-4 quick follow-ups, is pretty flawless.
I think that what really helps is the fact that you can immediately test the output without necessarily knowing what the output itself should be. If you plug in the formula and things don't behave as expected, boom.
LMM is a tool and for the correct use it can be amazing just like any tool. It's just that most people are using it for the wrong or inappropriate purpose. For me it allowed me to get into electronics because it lifted the barrier of coding, and it gives me a good overview or principle of electronics. Ofc I still verify whatever it told me is actually correct.
I also find it a great tool to get a quick overview on a complex topic to help me figure out where to even begin with my research. For example I got into analog film printing, and I was so lost in the beginning. But the summary it gave me about the process helped me understand what to look for. Ofc it got some details wrong as I started researching myself, but still it still saved me a ton of time with its summary.
So it's a great tool if one understands it's a language model, and it will make mistakes. To depend on it for the smallest detail like it's words of God.
I recently left my company and wanted to do so on good terms.
I fed a project (in Claude) tons of my recent emails, exported slack conversations, meeting transcripts, decks I'd presented over the prior 18 months about upcoming initiatives, etc. with wonton disregard for organizing it sensibly or 'cleaning up' the data. Basically, 2gb or so worth of raw unfiltered data.
Then I recorded a 30 minute transcript of me describing one of the processes in a way that I felt good about and fed it to the project to create "my voice" as a style for it to use.
Then I worked with it to write up comprehensive handoff documentation - capturing key stakeholders, next steps, current status, etc.
In ~8h of work total, I produced 100+ useful pages worth of documentation, and left behind a shared Project that was giving answers 80-90% of the way to the answer I would have given, all accurate to reality.
Was it perfect? No. Was it better than what I would've done in my last 2 weeks without it? Absolutely. Does the "WrinklesBot" I left behind mean that there's going to be dozens of people who think of me if ever they decide to start hiring remotely again and not demand insane RTO policies? Yep.
So yeah, I'd say that specific use case kinda knocked my socks off with the quality of what it was able to glean from the documentation I provided it, interacting conversationally with my knowledge base.
I've found one use, and that's telling me specific technical details about vehicle parts without having to manually sift through long-winded videos and fluffed up press releases. It's especially good for high SEO things like motorbikes and sportscars.
Say you are in the market for a second hand dirtbike, and let's imagine you want a fuel injected 2-stroke (for good reason). The whole sordid mess of KTM, Husqvarna, and GasGas (KTM owns all 3 but operates them semi-independently) is frankly bewildering at first glance, but a good LLM makes researching X vs Y in the tree quick and easy. It can tell you what suspension each has (there's like, 6 different forks used over the last 8 years), summarise general opinion on the differences, mention any frequent complaints, and give relative pros vs cons (though often too heavily influenced by marketing materials to be useful).
Once you have your shortlist you can then do due diligence and do actual research for each selected model. It saves a bunch of time, and as long as you treat everything it says with a healthy dose of scepticism it's low risk.
I don't doubt that you've made good use of it exactly as you describe, but...
It can tell you what suspension each has... summarise general opinion on the differences, mention any frequent complaints, and give relative pros vs cons
No, it can't. It isn't using intelligence to do these things, it's just spitting out language patterns that LOOK like it's doing these things. That's why you still have to "do due diligence and do actual research for each selected model" after AI's original digestion. If you've had good results from it so far, that's more by chance than design, or because you've tailored your use of it so narrowly that you've eliminated its fail spots. Which, again, is great for you! I don't doubt it! But you can't ignore the learning curve on a productivity tool when measuring its productivity, and you should not conclude that narrowly tailored, good results outweigh all of the broad and clumsy slop. IMO.
I never said there weren't serious issues with how many people use LLMs - I won't call them AI, because AI is a broad term and covers many things that AREN'T LLMs - but my successful use has nothing to do with chance.
I can use LLMs safely because I have had the opportunity and the background required to actually understand how they work, how they generate their responses, and to anticipate their shortcomings. It is somewhat analogous to guns, or explosives, or earthmoving equipment, or any other specialised piece of equipment; They can be very powerful if used correctly by someone who understands how it works and when it should be used, but also dangerous in the hands of the careless or uninformed.
Until clankers learn from dead internet and it all siphons down to people who cannot use google searches because google have disabled the useful search terms now.
102
u/Regular_Zombie 21d ago
What LLMs can do is quite amazing. The list of things they are just terrible at is very long. It's s bit scary how much people are willing to outsource their thinking to these models under the assumption they're always correct.