I am really concerned for the "LLMs don't work and are completely unreliable" folks

@reece I kind of get it. LLMs are useful, but there exist no guarantee that any llm output is "true" in any sense. But they almost always seem true (because that is what they are actually built to do - imitate). So care and scrutiny is always required. But a large subset of people believe actually no verification is needed and we can just automate everything with LLMs - dumb, but also dangerous.

@reece
When checking the generated text, how do you keep track of that you've vetted and what still needs checking?
How do you stay vigilant when hallucinations are rare?

@urbanmv The thing is, I don't think there is a large subset who think you can do whatever. Anyone not living under a rock has heard about the limitations.

@swope I think it fundamentally requires being a diligent thinker, no different than in the pre-LLM world honestly.

@reece @urbanmv Fundamentally they are not built to deliver "truth" - only what's "probable" - at it's base it's just spicy predictive text.

I've found my use of LLMs became far more useful when I shifted from trying to "single-shot" the prompt to get the "perfect" response over using it to generate possibilities.

Less "write this email for me" and more "here's what I'm thinking, what am I missing? OK, I like that bit, but not that bit. Would it be stronger if I said X?".

@reece at least three people I work with seem to think code reviews are "no longer necessary". One "staff engineer" told me sincerly that "tech debt no longer matters" and we shouldn't care about code quality anymore.

Though "loud subset" is perhaps more accurate than "large subset"

Even If there is only a 1% chance an LLM makes up random bullshit, then it my responsibility to fact check it 100% of the time I use it just incase.

If I have to fact check something 100% of the time then I don't see how you can't not consider it "completely unreliable"

@mattkirbylondon Yeah. Its great for exploring a space! Way better than trying to find a relevant Google query

@disaster2life Sure, but like . . . Google has limitations too!

@urbanmv There have always been ideological / irrational people