LLM’s can’t program.

https://functional.computer/blog/llms-cant-program

Alternative title: “How Samir got himself fired".

@samir

I recently realised that that’s why. Everyone else is. I have nothing to add, except that I find it personally distateful.

right. like literally once you've had people you trust do the work to try it out and conclude it's shit, what does another voice do in the cacophony?

So I scrolled through the approval test code, and discovered that it was not actually running the application and taking screenshots, but instead generating some random nice-looking screenshots with code, rendering them as PNGs, and saving them to disk.

LOL. this is what happen when you let machines mark their own work.

$1000 in tokens for this shit? truly the future of software development?

i did all my testing for free.

@samir fwiw, i have been codesigning complicated code with chatgpt recently and it's a mixed experience.

for the bits i'm not sure about the structure of, it acts like a rubber duck that talks back. it will make suggestions, some of which are worthwhile and most of which are not. it's sloppy in the other sense and imprecise and it will willfully misunderstand you even if there's only one sensible interpretation. but provided you know the material really well, it's probably more useful than not. but it's definitely not answers on a plate, it requires real work.

when i've written the code and ask it to review, it's again much the same. you can safely ignore most of it or disagree and get it to change its stance. what's left isn't a bad review, but like i would have spotted most of these instantly if i just came back to it the next day with fresh eyes. again, possibly a net asset, but like i have deeply thought through this code's structure and the only mistakes that should remain are simple code things, not the ideas. i will probably keep doing this for complicated code that noone else will review for me and not much else.

oh and i write all the code myself, obvs.

@dysfun Makes sense! I can absolutely understand using them, and I can see some value (as you saw), but I shall not bother, myself. I’m glad it’s sort of working for you.

@samir again, depends what you're doing. i've spent a lot more time writing code and thinking than i have asking the bot. i actively try to not use it as much as possible when i can avoid it.

what it is is like a really highly skilled ADHD peer that is having a bad day. this is still a step up from the zero people i have to discuss ideas with and get a review from normally.

but again, usually i don't need much help because i am quite confident in my ability to produce software by now. i'm really only doing this because i'm doing novel concurrency work and i want to be sure i get it right and noone else is going to be there to replace the bot.

@samir also note that this isn't fashionable agents bollocks, this is not going to produce scads of code faster than i could write a few lines of code. this is considered design work by a skilled professional with ideas in mind that it's shrunk the time to perform.

in other words, this is all value it adds to an existing human that writes the code themselves and is prepared for it to take longer than an hour, not an excuse to lay people off.

@dysfun @samir

Generative AI Large Language Models (LLMs) lie and cheat.

And/but, given that everything they do is a reflection of their training data, I have to wonder …

Is this evidence that most of our industry is lying and cheating, and has been doing so for decades?

😱 😱 😱

I don’t have a soundcloud. Go donate some money to someone in Gaza, if you can.

https://gaza-verified.org/

@samir yes, this is my finding too, basically.

@samir Thanks for sharing your experience with chatbots; it gives me some ideas on what I should be checking for during code V&V.

On that note, you finish with "code review is stupid"; could you unpack that a little (or have you in a previous post)? I work with engineering safety analysis software and code review has always been a valuable part of the process. Maybe it's the engineering culture of peer review for error checking and constructive criticism, but I find the process supports my team rather than works against it. I'm curious why you don't feel it's worthwhile.

@arclight My guess is that you are much more rigorous about code review than most software developers. Everywhere I have worked, it’s been more of a gut check: reading a diff, pointing out some typos, then saying “LGTM”.

I believe strongly that pair programming is more effective at preventing bugs than lax code review.

I linked to the SmartBear book. I think it explains in greater detail the difference between real code review and the performative nonsense (pushed by GitHub) going on in many places.

@samir Thanks - I think there's a big cultural difference between engineering & software development on the practice of reviews. If the rigor isn't there, it's kind of a performative waste of everyone's time and it can be pretty corrosive if people make or take reviews personally. For us, it's just an extension of brick-and-mortar work practice - everyone makes mistakes, it's not a personal challenge, and it's everyone's goal to catch problems as far upstream as possible. It's another layer of defense in the Swiss cheese model of risk. But if it's not working as intended, yeah, you run the risk of having a false sense of rigor, larding up ypur process, and causing unnecessary conflict.

@arclight Bingo. According to the research I’ve read, good code reviews need at least: a full understanding of the existing code, an understanding of the requirements for change, explanations of why decisions were made, and for the change to be small enough to be fully held in someone’s head at once.

My experience with pull-request-based code review, across many organisations, is that it’s rare for any of these boxes to be checked.