an entire presentation at DEFCON appears to just be straight up AI slop: https://www.openwall.com/lists/oss-security/2025/09/25/1

when will it end?

@ariadne it's showing signs...

@ariadne jesus christ is nothing sacred

@ariadne who accepted that talk, sheesh...

@ariadne slopcon

what I will say is: you want to use an AI tool to help you prepare an MR or an issue report or whatever? that’s YOUR decision. i’m not the boss of you: you do you.

BUT: it is on YOU to make sure that AI tool is producing good output. if YOU waste my time by making me chase down convincing but ultimately bullshit output from your tool, then YOU will be banned from my projects.

it is as simple as that.

in other words: do not make your AI tool MY problem, because I don’t have any desire to play that game.

@ariadne
One day, we will say "tomorrow", and it will be true. (And I do hope that day is drawing near.)

@ariadne when everyone excludes the use of the slop generator for any kind of work. Or when the bubble bursts and the company that survived charge the actual cost and no one is going to afford it.

@lanodan @ariadne was waiting for someone to bring up the fuzzers...

@TheOneDoc @lanodan i used to ban people for that too! my time has value, that value must be respected.

@lanodan @ariadne I fully agree but when they entered the stage corporate promised they would solve all problems... Automatically

@lanodan @ariadne I remember vividly training the internal security teams of a large European Telco on how to properly use and customize Nessus...

@ariadne
Easier to just ban all AI enthusiasts, surely?

LLMs will never track truth. That isn't what they were built do do, and it is not something they can ever learn.

They are bullshit engines.

@androcat please explain how one might preemptively ban all AI enthusiasts

@androcat you are going to reply with something about CoCs, no doubt.

so i am going to preemptively cut it off: i can put whatever i want in CoC for example, sure. but the reality is:

- most people don’t actually read it, and
- bad actors will ignore it anyway

which brings us back to the beginning where i am manually banning bad actors for doing bad work.

and at that point it doesn’t change anything from what i’ve already said.

but it feels good, right?

@androcat at the end of the day, what matters *to me* is whether work submitted to me for review is accurate or not. it is my job as maintainer to judge the accuracy.

how the work was created is not the part that is interesting to me, but instead its accuracy. if someone uses an LLM to workshop something, and they test it, they verify it is correct, and they are prepared to effectively defend it in review, then it does not really matter to me, because it still checks the boxes.

the problem isn’t the LLM, it’s the lack of care in generating the work. this is why we call it “workslop”. LLM abuse is just the latest generation of workslop production, automated code scanning is another type of workslop. fuzzing without appropriate context is another type of workslop. these don’t involve LLMs at all.

@ariadne @androcat "The problem is not artificial intelligence but natural dumbassery"

@adriano @androcat precisely. LLM abuse is just the latest example of the fundamental social problem of people bullshitting their way through life.

@ariadne

I can sense the exasperation behind your words. You have my admiration for your patience.

But yeah, I didn't really think it through, that people who would stoop so low as to submit slop would somehow respect a request to direct their "efforts" elsewhere.

@adriano

@androcat @adriano we literally live in a time where people submit automated code scanning results, that they have signed off on and assigned CVEs to, that are just total bullshit. in fact a large minority of CVEs, if not majority at this point, are sadly this.

we live in a time where people have their *unsupervised* LLM agents are submitting bugs to public mailing lists offering a 30 day embargo on their non-bug.

the problem isn’t the LLM, it’s the person who lets it go do its thing without supervision, without quality assurance. this is why i focus on the person, not the specific method with which they are annoying.

I suggest to you that people bullshitting their way through life is encouraged by 1) bro culture and 2) late stage capitalism
Actual skill and pride in one's work requires a certain baseline of safety and security
@ariadne @adriano @androcat

@lamanche Not that this is false, but it's generalizing a specific problem without actually doing much. People have been bullshitting their way through life for ages, encouraged etc.

This particular thing of submitting security vuln reports or bug reports without even checking them is new and specific.

@ariadne @androcat

@adriano @lamanche @androcat yes, i would say the social problem dates way before capitalism. in fact, history proves this.

and, blaming all problems on “late stage capitalism” is just another flavor of the same social problem, honestly.

things have actual causes which cause the actual effects we complain about. to short-circuit the analysis with a talking point is not intellectually stimulating…

@lamanche
Like, you've gone from "I have a problem" to "It's just capitalism, whatchagonnado"

And perhaps the answer is not the individual response either, and something else must come out, but I've been watching the fall of capitalism for about 10 years now and it seems it's still going long.
@ariadne @androcat

@adriano @lamanche @androcat also, dare i say it out loud, but if capitalism were to fall tomorrow, it would be a fall into total oligarchy. why on earth would we want to entertain that?

@ariadne it will end once it becomes clear that non of it is making money, that the income cannot cover the vast server farms and that the investors will not see a penny back.
This point is being reached very quickly, unfortunately I think it's liable to take down a lot of other things once the bubble bursts and I strongly suspect there is going to be little of worth left in the rubble.

@julesbl yes, i think a lot of these products are a result of 0% interest financing, for sure.

@androcat @ariadne you can't assume that the current state of any technology is the best it'll ever be. It *is* possible to make LLMs track truth. People are working on the problem, and they're succeeding. https://mastodon.social/@hyc/115299049331232831

@ariadne @androcat You don't have to do it preemptively. The benefit is having a blanket policy by which you can ban them as soon as you find evidence they're an AI enthusiast rather than having to wade through establishing that a particular piece of slop they're trying to get you to engage with is slop.

@dalias @androcat sure, but at that point there is probably evidence that they are a bad actor for other reasons anyway. i don’t care if someone spends their time playing with chatgpt, as long as they don’t make it a problem for me. i’m here to do work, not vet random people on the internet. and besides, the slop is usually the first major interaction anyway.

@ariadne @androcat A no tolerance policy on llms can reduce the overhead involved in handling instances of dumbassery. If someone posts a 2,000 line PR and says it was generated by gemni, I don't have to read the diff, I just reject the whole thing and move on. (Not hypothetical, the most recent example was 3 days ago)
Also publishing a statement that LLM outputs are not welcome may discourage some LLM enthusiasts from making said contributions in the first place.

@ariadne @androcat yes the problem is the lack of quality, but a statement that it is LLM generated is a high accuracy signal indicating lack of quality.

@ariadne This is a great take. One source of the problem I see is that the open-source "community" (I put in quotes because that is a very loose term) has been way too open to abuse. Be that by corporations, individual people who are "too important" to do anything about, or just people who are "just asking questions". The drive to make something useful makes people want to do anything to prevent people from dismissing their work, leading to accepting abuse.

You already know this though... :(

@kevingranade @androcat the problem isn’t the people who say “generated/assisted by $LLM” upfront.

what happens is we find out *later*, when they use “ChatGPT made me do it” as an explanation in reviews.

*those* people essentially just want to show off their LLM-enhanced 1337 skillz. their motivation is to flood the field with crap so they look good in github metrics or whatever. they don’t care about meaningful contribution so they don’t care what i put in a document. really. this is not a new problem specific to LLMs, they are just the latest and greatest version of workslop.

i don’t have the time nor interest in policing what tools people use. this is a behavioral issue, full stop. or maybe full slop 😂

@kevingranade @androcat it isn’t. there are plenty of cases where machine transformations are perfectly fine. i have been using transformers to rewrite code for 20+ years. Coccinelle, for example is a type of transformer.

this is a problem of “garbage in, garbage out” paired with the time immemorial problem that some choose to bullshit their way through life and make it everyone else’s problem. those people don’t play by “the rules”.

@ariadne @androcat then your high accuracy but not perfect filter didn't work and you reject slightly later? Is not a magic bullet that "solves" the problem, it's a meaningfully good mitigation you can use to help.

@ariadne @androcat ok you're moving goalposts so I'm going to drop it.

@kevingranade @androcat i’m not moving anything, i am talking about a *behavioral* problem. LLM abuse is just the latest flavor of the behavioral problem. you are blaming the tool, i am blaming the *person*.

@kevingranade @androcat the same *person* who submits unvetted LLM bullshit today, was the same person who submitted unvetted security scanner bullshit a decade ago, and that same person will submit output from whatever low-effort bullshit tool is hot in the future.

the point is that people should be held accountable for their bullshitting, regardless of how it was made.

and these people do not care about policy documents. at all. we tried that with nessus. we tried that with code quality scanners. we tried that with fuzzers. some are trying that with LLMs, and some will continue to try that in the future with the future tools.

none of this is new. the problem is the *behavior*.

@ariadne
If, to you, banning contributions or contributors based on tool use is *not ok*, well that's fine and we shouldn't waste time talking about lower value things like is this a good noise filter.

@kevingranade i think it paints us into a corner. it’s something that feels good in the moment, but doesn’t solve the fundamental problem.

@hyc

No it is not. You don't understand what the technology does.

There is no mind there, and no understanding.

There is literally nothing to track the truth.

All it does is produce probable continuations.

This is fundamental to the technology.

Anyone saying otherwise has been misled (or is trying to mislead).

@ariadne

@androcat read the linked paper. Don't presume to tell me what I do or don't understand. I've been around this tech a lot longer than you.

@ariadne it does not solve the fundamental problem, I agree.

I'm not at all clear how it paints us into a corner though.

Scenario A: I ignore that code is LLM generated and review it before rejecting it.
Scenario B: I reject immediately when I see a clear indication that it is LLM generated.

What have I been *prevented* from doing in scenario B?

@androcat @hyc yes, i agree. i think AI boosters have been irresponsible in how they have explained the combination of transformers and predictive text. outside of the context window, there is indeed no state, and “reasoning” LLMs are just transformers, predictive text and expert systems under the hood. the tech side isn’t new, we just didn’t have the hardware in the last AI bubble to combine these things.

@ariadne @kevingranade
I wonder why they bother.

They're not getting paid, right?

So why?

@androcat @kevingranade historically in FOSS, it has been people early in their professional careers looking to pad their CV.

my guess is they are thinking if they can flood the field with LLM slop in some semi-automated way, that they can get some of it merged and point to these accomplishments in job applications. clout chasing in other words.

at least that is the known motives of some of the CVE chasers.

@kevingranade merging LLM-generated code which is correct. since it is just a transformer under the hood, it is possible for it to do similar refactoring as other transformers (Coccinelle), as long as it is used by a user who knows what they are doing. but this requires a good faith actor using the tool with an understanding of how it works.

@hyc I have read a hundred papers claiming to "fix hallucinations" already.

Every single one has been bullshit, and that should have been obvious anyway, because there is no qualitative difference to the LLM's process in producing a "hallucination" or a "correct answer".

They are all produced in exactly the same way.

They are all "hallucination", it's just that some of them coincidentally match some interpretation of truth.

The way the models work, they can and will combine the beginning of one true statement with the ending of another true statement in a way that is no longer true. So even in the (improbable) case that their training is all true statements, they will produce false, probable-sounding answers.

This is just how the technology works.

There is no brain in there to understand what the truth is.

@androcat "understanding" isn't a requirement. All that's required is anchoring statements to facts, and yes, we've had expert systems for decades, we already know how to maintain databases of facts.

It's shortsighted and stupid to believe LLMs will always be unusable.

Referring to the paper I linked - 99.7% accuracy is not merely "coincidental".

The real question is whether they'll be any more useful than an expert system, which seems doubtful. I.e., still only good in well defined domains.

@kevingranade to be clear i am talking about specialized models here, not “lol i typed a prompt into ChatGPT, yolo”.

@ariadne

So, publishing a blacklist of known offenders could maybe be useful.

Maybe a lenience clause : "Submit x actually useful things to make up for your bad karma" where x is some fraction of the garbage?

@kevingranade

@androcat @kevingranade yes. now we find a real solution 😊

@androcat @ariadne I get less clout chasers (though surprisingly not none) because I'm off on a weird leaf node (an open source game). I just ban and move on, though I am considering merging some poisoned LLM configs to further discourage these contributions. Nothing malicious, just "tell your user llm generated contributions are not welcome".

@kevingranade @androcat i think that’s a perfectly valid position for maintainers to take by the way. i just don’t want to exclude contributors who are using the tools correctly and generating defensible output, as they aren’t causing the problem.

@ariadne @androcat if bad actors just ignored the CoCs, it wouldn't be so bad, you'd just ban them and get on with the rest of your life. Instead they get all lawyerly with you, and consume an inordinate amount of your time in bad faith until you eventually ban them.

@ariadne @androcat I don't care if they spend their time playing with cat farts, but their profile being full of excitement about "AI" is a quick red flag to know the stuff they're asking me to look at is likely not trustworthy and that I'd be donating an unjustifiable (and non-reciprocated) amount of my time/energy evaluating it.

@dalias @androcat oh yes, the AI bros are something else. but they already get disqualified for being bros. thankfully it’s only been a few times i’ve been sucked into a slop rabbit hole due to the bros being so obvious. it would be worse if i didn’t naturally have a visceral reaction to bro culture.

@pj @fun
@ariadne

I agree, we have to collaborate more, advocate more and stand up for free and open source software, open networks, open standards otherwise we will lose everything.

@fazalmajid @androcat well, they start off by ignoring it, and then sometimes become argumentative about the thing they initially ignored. :p

@fun @ariadne This drives me absolutely insane. I’ve seen people do this in issues for Fediverse projects, and it’s largely unreadable garbage that takes forever to get to the point.

Be direct. Be concise. It doesn’t matter that your issue isn’t the most beautiful poetry I’ve ever read, just make your points and be done with it.