@mjg59 you mean "not by paying monthly $200 to a wanna be megacorp"? Yeah, not like that indeed.

13 years old me started coding on an old Windows 3.1 workstation with ~$0 monthly cost. If I were to enter the industry now, when one has to invest in LLMs, which btw also prevent from gaining actual skills and erode existing skills, I would simply have not done that. Must be why genZ hates LLMs

I don't see how one can look at the thought-extruding machine and think "surely it will liberate me"

@mjg59

I'm a full-time professional novelist. Have been for 25 years. Before that I was a software dev. From the inside, the cognitive experiences of writing prose fiction and writing software *feel identical*. The creativity exists outside the words, and most of the phrases and grammar I use are unoriginal.

Ball's back in your court.

@foxyoreos @petko I think merging code people don't understand is a terrible thing no matter what! But there's a meaningful difference between "I am a project maintainer and I care about this being maintainable" and "I am a consumer and I just want this fucking thing to do what I want". We are already at the point where local agents can help with the latter without any ongoing SaaS or giant data centre requirements

@mjg59 @petko you're still focusing on the individual impact of singular merges. I'm saying that the consequence of promoting this technology is that code is going to get merged into production (including into open source projects) that isn't understood.

That would be the case even if only local models existed. There's no world where this is only used on local projects.

And the end result is that it will be harder to read a bunch of code, and those codebases will be less accessible.

@foxyoreos @petko The existence of Javascript runtimes made things worse for a whole lot of people and also made things better for a subset and urgh every single thing we do to reduce barriers potentially allows worse outcomes and the solution isn't to stop lowering barriers it's to keep having strong project maintenance

@foxyoreos @petko This is the "Visual Basic is bad, actually" space which is something I do actually have empathy with and also I think the world would be a worse place now if the gatekeeping there had been effective

@mjg59 @petko I'm sorry, do you think the problem with AI right now is that it *reduces barriers*?

That is a million miles away from my complaints with the industry.

@foxyoreos @petko Then you're complaining about something I wasn't saying

@mjg59 @petko AI is not Visual Basic. This is absurd.

If AI just made programming accessible, I would have no problem with it. Visual Basic did not create a RAM shortage. Visual Basic did not lead to open source developers closing their issue trackers or shutting down bug bounties. Visual Basic did not cause a resurgence in coal power. It was never used to help bomb a children's school. It did not lead to mass disinformation. It was not used to justify layoffs. Etc, etc, etc.

@mjg59 @petko I am saying that the usage you are advocating for cannot be disentangled from these outcomes.

While we are headed on the trajectory we are headed, usage of these tools even for local projects, fuels our descent into a world that will be worse for everyone *including* new programmers and hobbyists.

@foxyoreos @petko The scenario I am describing is "A user wants their software to do what they want". That is a desirable outcome! There are scenarios where that can happen without attachment to commercial subscriptions or massive energy expenditure, and if your argument is that this is a slippery slope that enables other bad outcomes then I think you're actually making those bad outcomes more likely

@mjg59 @petko there is no way right now to get the good without the bad. Maybe once the bubble collapses that might be different, and I'm willing to talk about things like local models then.

In the meantime, we should not be using this stuff period, whenever it is possible for us to avoid it - even for hobby projects, and even using local models. I'm happy to revisit that once the house fire is put out.

@foxyoreos @petko "I refuse to use this technology in an ethical manner because others are using it unethically" is incoherent

@foxyoreos @petko Fight against the unethical cases! Point out the direct harm that exists as a result of what people are advocating! But this is like saying that nuclear energy is unethical because people can do it badly and cause massive environmental damage

@mjg59 @petko I'm not arguing that it's a slippery slope, I am looking out my window and looking at news articles and directly seeing right now, that local projects are being used to justify enormously harmful outcomes.

There's no slippery slope here, there's just stuff happening, right now.

We absolutely should push back on accessibility tech that is more harmful than helpful.

@foxyoreos @petko Refusing to use a thing that can be used for good because it is also being used for bad is a terrible position to take

@foxyoreos @petko This is giving up social media because fascism is able to use it effectively. This is giving up newspapers because fascism is able to use them effectively. Using this to make the world a better place isn't ceding the argument that it can be used in ways to make it worse.

@mjg59 @petko it is very hard for software engineers I think to understand sometimes that they are not islands.

It can be frustrating that in a given climate an otherwise innocuous use of tech is problematic, and I get that. I am also sympathetic to that, but it's just what reality is right now.

@foxyoreos @petko I can absolutely promise you that we are already so far down this fucking rabbit hole that refusing to use this tech at all is going to result in no good outcomes without preventing any of the bad outcomes

@mjg59 @petko boycotts work when they are clear, easy to understand, and don't have a giant list of exceptions.

It's not aesthetic, but it's practical. You make a simple policy that can be easily understood and you stand by it - that is how you practically build a consumer movement against something.

This is just what organizing looks like. And large backlashes against AI, the aesthetically ugly "unnuanced" takes are, at the moment, infinitely more practical than what you're proposing.

@foxyoreos @petko The luddites weren't boycotting technology as a whole, they were boycotting technology in a way that hurt them. Can you point at any historical case where boycotting a technology as a whole has resulted in a better outcome? The only ones I can think of have made things worse.

@mjg59 @petko no, you weigh the pros and cons.

You decide on a case by case basis whether complete abstention helps more than it hurts. Not everything in the world is the same, some things are more harmful than others.

Also.. not everything is entrenched to the same degree, some things are easier to boycott.

This is how all boycotts work.

@foxyoreos @petko Right. And in this case complete abstention alienates everyone who understands that this can be used in an effective manner without fucking over everyone else. You're losing your most effective proponents.

@mjg59 @petko I very firmly believe that mass public backlash against this technology across the board will at the very very least reduce harm.

And I know that because I am also seeing it happen in the real world.

Games promising not to use AI in development. Advertisers pulling back on campaigns. All of these movements are happening because of hardline anti-AI positions. This stuff is practical and it works to reduce some of the harm being done.

@foxyoreos @petko Presenting this in an entirely unnuanced way means that everyone who has experienced the ways that it can be used to improve things without fucking everything up are no longer on your side. You're fighting against calculus because calculus enables ICBMs.

@mjg59 @petko I'm not asking to boycott technology as a whole, what on earth do you mean?

I'm not even asking to boycott all AI - just the currently hyped collection of GenAI and LLMs. Heck, I'll even throw you a bone - if you want to use a local LLM trained on ethically sourced data trained not by a company like Facebook, go wild.

That doesn't exist in a useful way for programming of course, which should tell you something about the tech, but..

@foxyoreos @petko It absolutely does exist in a useful way for programming!

@mjg59 @petko The luddites are very directly comparable to this.

I'm not telling you to boycott dwarf fortress, we are talking about a very small subset of AI that necessarily today involves unethical training to be useful for coding.

@foxyoreos @petko No, that's not what you said at all. If you want to redefine your original argument do so explicitly, don't say that this is what you always meant - I was explicit about what I was advocating for

@mjg59 @petko do you have polling to back that up?

Or am I just losing specifically you?

Again, these "effective proponents". They're not the reason McDonalds pulled back on AI ads. They're not the reason that I've seen game devs pull back from using genAI even for concept art. They're not the reason there's an advertising push for AI free products.

That's the hardliners.

Are you calling senators about this every day? Who exactly am I losing?

@foxyoreos @petko AI ads add nothing to anything. AI gamedev work takes creative endeavour away from workers while making things worse for consumers. Someone making software do what they want it to? What are you proposing as the alternative? They just don't get to do that? You have an (understandable!) kneejerk reaction to this technology because of all the awful ways it's used, and now you reject the good ones as well.

@mjg59 @petko well, opposing local usage of an LLM for programming is not the same thing as universally opposing a category of technology. Opposing LLMs entirely is not the same thing as that, any more than opposing NFTs if the same as opposing all cryptography.

That's all I mean. We're talking about something narrow here, a subset of AI with particularly harmful effects.

@foxyoreos @petko What is the harmful effect of local LLMs used to modify local code that is not published?

@mjg59 @petko genuinely in what way?

I'm not aware of any ethically sourced dataset large enough to train an LLM that would be effectively writing code.

I mean if one exists - if you can build one without training a base model on author works, or mass scraping GPL code, or mass scraping websites.. I guess go for it? I still have qualms, but sure, I'll call that separate from the industry.

But like, be real - is that what *anyone* is using?

@mjg59 @petko is there data to back that up?

AI opposition is currently at the strongest it's ever been, at the same time that many people are taking hard-line positions against genAI everywhere.

Correlation is not causation of course, but if what you're saying is true, we should be expecting AI opposition to go down, right? But it's not going down. And from what I can see online, where I do notice a decrease in opposition is not with ordinary users, it's with *specifically programmers*.

@mjg59 @petko but.. I mean, not to be dismissive, but I would much rather win ordinary people and lose programmers, rather than the other way around. I don't see the programmers, to be frank.. doing much right now.

I see a lot of ordinary people protesting data centers, calling school boards, boycotting games and TV shows, etc..

@foxyoreos @petko I think we're approaching this from wildly different directions while desiring the same outcome. I do not want giant data centres. I do want people to be able to use LLMs to their local benefit. These are 100% compatible positions. If you object to the environmental outcome (which you should! I do!) that's the thing to campaign against. We can benefit from LLMs without that harm.

@foxyoreos @petko To the extent that we have legal precedent on this matter, training on GPLed material is legal. As a former FSF board member, I think this is the correct outcome.

@mjg59 @petko this comes up in environmental movements too - a lot of people have theories about what the best ways are to protest, but then you look at the data, and loud disruptive hardline protests are just.. more effective. It's counterintuitive, but practical. Practical mass movements are exercises in simplifying concepts and teaching people to be loud.

@foxyoreos @petko Protest the data centres! Protest the gas turbines! Protest the fossil fuel impact! I can use an LLM to transcribe video on my laptop without contributing to that!

@mjg59 @petko

> What are you proposing as the alternative?

Communities! <3 They work, they're empowering, they're emotionally healthy and satisfying, and they lead to better results.

The alternative is mass outreach to communities (not software engineers, people *in* the affected spaces), and building support networks to plug into.

I do this already with software, I have multiple forks of FOSS programs that are used by like 3 other artists like me. Communities have zero of the downsides.

@foxyoreos @petko Communities don't write code without people in those communities writing code! We don't have anywhere near the resources for people who know how to write code to do so for everyone who has a desire for software to behave differently - it works within a given community, but we have no way to scale that globally

@mjg59 @petko

For the user:
- deskilling (I don't mean this in a derisive way, this is not like Unity or visual programming, LLMs literally decrease user performance over time)
- isolation from communities they are a part of

On the higher order:
- All of these LLMs (that I'm aware of) are trained unethically on mass unlicenced tech and writing
- Popular local LLMs are reliant on large companies to produce them, and this creates a dependency on the very companies we're trying to avoid.
[...]

@mjg59 @petko [cont]

- their popularization is being directly used today as propaganda to excuse more harmful uses of the tech (I don't mean this as a slippery slope, I mean it's actively happening now).

Now, I know you disagree with some of those higher order negatives. Fine.

Do you have an objection to downsides like deskilling? (again, this is entirely different from JS or Unity, which do not have the same mental effects, it's more comparable to, if anything, lead).

@foxyoreos @petko I think having people who have no skills to begin with use these tools in specific ways does not result in deskilling

@mjg59 @petko on the meta level, I also do not believe it is possible to advocate right now for local usage without causing harm.

This is a different objection than local usage itself.

If someone privately uses an LLM locally but doesn't say anything about it, I think that's less harmful than going online and advocating that they should be used or encouraged locally.

I recognize that you also disagree that should be an objection I have, all I can say is I see what's happening online.

@mjg59 @petko I think that normalization is a harm that the programming industry and programmers generally have not grappled with in a serious or rigorous way.

@mjg59 @petko I do not think *any* of those downsides apply to Javascript, Electron, Visual Basic, Wordpress, etc. I think these are harms that are (mostly) unique to LLMs.

@mjg59 @petko okay, this is not worth fighting over, I'm not going to argue the GPL - even if you include GPL code (and I will also note that legal and ethical are not the same things) - but let's say all open source code is fine.

You can't build a programming LLM on just open source code, because programming LLMs are not built on just code. They're built on massive amounts of non-code text as well.

I take a stricter stance on the ethics of training on mass code, but I recognize many don't.

@mjg59 @petko I mean, that's the conflict. Where did that LLM you're using for transcription come from? And are you going online and arguing that the people who oppose normalizing that usage are opposing accessibility?

I don't think it's as simple as just saying "well my usage is separate."

I mean, it's already not separate for you, you're going online and arguing for it. That has impacts on the ecosystem, even if you would prefer it doesn't.

@mjg59 @petko I'll push back on that. I think we do have the resources to embed programmers into communities and train individuals within communities at scale.

I'll make a bolder claim. The impact I have seen from even non-LLM AI on accessibility is regularly negative when it is used as a substitute for communities.

YouTube transcripts getting worse with community tools getting replaced by AI. AI alt text is regularly called out by accessibility advocates. And neither of those are LLMs!

@mjg59 @petko there is a growing body of research that heavy usage of LLMs results in *general* degradation of attention and perseverance when faced with new tasks. I'm not talking about getting worse at programming, I mean there's a good chance using these things heavily makes you worse at everything.

That's why I compared to lead more than anything else. I don't think the effect is the same (LLM negative effects are almost certainly going to be less permanent), but..

@wouter Existence is a prerequisite to using something. You can't use something before it's been made.

So if you want to advocate for a theoretical future ethical LLM, be my guest. I encourage it. But that doesn't excuse using an unethical model today.

@wouter he's advocating for people to use it today.

Not tomorrow. Not to use a theoretical version that might exist. To use the existing local models, today.

The moment you create one of these theoretical ethical models, let me know. But I guess I missed the part of Linus Torvalds story where he said, "I can envision a free operating system someday, and therefore I'm gonna start using Windows now."