A user asked on the official Lutris GitHub two weeks ago “is lutris slop now” and noted an increasing amount of “LLM generated commits”. To which the Lutris creator replied:
It’s only slop if you don’t know what you’re doing and/or are using low quality tools. But I have over 30 years of programming experience and use the best tool currently available. It was tremendously helpful in helping me catch up with everything I wasn’t able to do last year because of health issues / depression.
There are massive issues with AI tech, but those are caused by our current capitalist culture, not the tools themselves. In many ways, it couldn’t have been implemented in a worse way but it was AI that bought all the RAM, it was OpenAI. It was not AI that stole copyrighted content, it was Facebook. It wasn’t AI that laid off thousands of employees, it’s deluded executives who don’t understand that this tool is an augmentation, not a replacement for humans.
I’m not a big fan of having to pay a monthly sub to Anthropic, I don’t like depending on cloud services. But a few months ago (and I was pretty much at my lowest back then, barely able to do anything), I realized that this stuff was starting to do a competent job and was very valuable. And at least I’m not paying Google, Facebook, OpenAI or some company that cooperates with the US army.
Anyway, I was suspecting that this “issue” might come up so I’ve removed the Claude co-authorship from the commits a few days ago. So good luck figuring out what’s generated and what is not. Whether or not I use Claude is not going to change society, this requires changes at a deeper level, and we all know that nothing is going to improve with the current US administration.
Are you talking about his way of communicating or about his AI use? I think it could have been said a bit more level headed, but I mostly agree with what he’s said. I also see no issue with the part “good luck finding it then” that seems to sound malicious to you. To me this means “if you can’t find a difference in quality, your whole complaint is invalid because there basically is no difference in quality”. Yes, it’s still AI and should not be viewed as more than a knowledgeable intern, yada yada, but I hope the point comes across…
If there’s no difference in quality why obfuscate it? Why hide something that you think is a valuable tool if your code can speak for itself?
He could have used that opportunity to take a standing his own way “this is what I am doing and if you don’t like it feel free to make a fork but I think this is blown out of proportion for: (reasons he could list his opinions on)”
But being like “good luck finding it now” is 100% malicious in this context. Or if malicious is too strong of a word for this, its definitely not user friendly at all.
And certainly not very “open”.
I don’t see it as obfuscation if there is no underlying difference. Why treat working code differently depending on the source if what matters is that it works (which it does by definition). Of course there has to be more quality control if AI is able to produce more code, but I don’t think that’s the point here right? Why highlight the different sources of the code if, as you said, the code can speak for itself. What’s the difference to you if you can’t tell them apart?
The difference is that AI is a known issue creator (that huntarrr app comes to mind) with many projects and AI usage is supposed to be disclosed transparently for compliance with copyrights and licensing.
But even despite all that its kind of a shitty way to go about it the way he did, in my opinion.
The timeline was that he started adding attribution indicating the use of AI.
Then the anti-AI drones started bombarding the Github, Discord and forums with harassment. His recent statements and removal of attribution are entirely addressed at and because of the anti-AI people harassing the project staff.
He’s not removing it and saying ‘fuck you’ to the users. He’s tired of being harassed by third parties who are not involved with the project in any way and so he removed the source of the harassment.
In my opinion, he should’ve left it as a co author. I think if you as a user have an ethical issue with Claude, that’s your choice and you can make the decision not to use lutris. I mostly agree with what he says until that part about removing Claude so “good luck finding it”.
It’s not about finding a difference for people (usually), it’s about how that model was trained on the work of others, without consent, for free, to then sell. He made his points about how much it helps, that it’s better than using Meta, Google, OpenAI, or Copilot and I think that’s probably true. But he made that case, so why then hide what Claude has done?
In gaming, Valve requires you to list if you have used AI in the creation of your game and you describe in what way. It’s not because the game will 100% of the time be absolute slop (right now it usually is), it’s so that the potential customer can be informed and choose to or not to support the use of AI in those products.
As far as I’m reading, most people who reviewed the actual code think it’s fine. So, again, I don’t see the point in hiding it other than being somewhat petty.
The point in hiding it was that it was being used, without harassment or complaint, right up until he added attribution which resulted in an avalanche of complaints which require resources to deal with. Discord, the forums and Github pull requests now require much more moderation labor, which takes away from the project.
People had no complaints about the code quality until he started adding AI attribution. So he removed the attribution.
Like he said, if people can’t tell the difference until he started marking the code AI assisted… then they don’t actually have an argument and are simply bringing anti-AI politics into the project.
Right, fair point with the training data.
Think of it like a jeweller suddenly announcing they were going to start mixing in blood diamonds with their usual diamonds “good luck finding them”.
Functionally, blood diamonds aren’t different.
Leaving aside that you might not want blood diamonds, are you really going to trust someone who essentially says “Fuck you, i’m going to hide them because you’re complaining”
If you don’t know what blood diamonds are, it’s easily searchable.
I’ll go on record as saying the aesthetic diamond industry is inflationist monopolist bullshit, but that doesn’t alter the analogy
Secondly, it seems you don’t really understand why LLM generated code can be problematic, i’m not going to go in to it fully here but here’s a relevant outline.
LLM generated code can (and usually does) look fine, but still not do what it’s supposed to do.
This becomes more of an issue the larger the codebase.
The amount of effort needed to find this reasonable looking, but flawed, code is significantly higher than just reading a new dev’s version.
Hiding where this code is makes it even harder to find.
Hiding the parts where you really should want additional scrutiny is stupid and self-defeating.
Thanks, I think your first point is a really valid one. AI technology is far from clean, especially in a political scope.
To your second point. I see that, but on the other hand, it makes an impression on me as if human code would be free of such errors. I would not put human code on an (implied) pedestal (especially not mine), but maybe I’m missing your point. I think being suspicious about AI code is good but same goes for human code. To me it sounds like nobody should ever trust AI code because there can or will be mistakes you can’t see, which is reasonably careful at best and paranoid at worst. At some point there is no difference anymore between “it looks fine” and “it is fine”.
Let’s assume we’re skipping the ethical and moral concerns about LLM usage and just discuss the technical.
Nobody who knows anything about coding is claiming human code is error free, that’s why code reviews, testing and all the other aspects of the software development lifecycle exist.
Nobody should trust any code unless it can be verified that it does what is required consistently and predictably.
This is a known thing, paranoia doesn’t really apply here, only subjectively appropriate levels of caution.
Also it’s not that they can’t be seen, it’s just that the effort required to spot them is greater and the likelihood to miss something is higher.
Whether or not these problems can be overcome (or mitigated) remains to be seen, but at the moment it still requires additional effort around the LLM parts, which is why hiding them is counterproductive.
This is important because it’s true, but it’s only true if you can verify it.
This whole issue should theoretically be negated by comprehensive acceptance criteria and testing but if that were the case we’d never have any bugs in human code either.
Personally i think the “uncanny valley code” issue is an inherent part of the way LLM’s work and there is no “solution” to it, the only option is to mitigate as best we can.
I also really really dislike the non-declarative nature of generated code, which fundamentally rules it out as a reliable end to end system tool unless we can get those fully comprehensive tests up to scratch, for me at least.
Thanks for taking the time to reply.
Greater compared to human code? Not sure about that, but I’m not disagreeing either. Greater compared to verified able programmers, sure, but in general?..
I don’t think I’m getting your point here. Do you mean by that, the code basically lacks focus on an end goal? Or are you talking about the fuzzyness and randomization of the output?
Both.
The reasons are quite hard to describe, which is why it’s such a trap, but if you spend some time reviewing LLM code you’ll see what I mean.
One reason is that it isn’t coding for logical correctness it’s coding for linguistic passability.
Internally there are mechanisms for mitigating this somewhat, but its not an actual fix so problems slip through.
The latter, if you give it the exact same input in the exact same conditions, it’s not guaranteed to give you the same output.
The fact that its sometimes close to the same actually makes it worse because then you can’t tell at a glance what has changed.
It also isn’t a simple as using a diff tool, at least for anything non-trivial, because it’s variations can be in logical progression as well as language.
Meaning you need to track these differences across the whole contextual area which, if you are doing end to end generation, is the whole codebase.
As I said, there are mitigations, but they aren’t fixes.