That would require LLMs to truly learn, which they don’t. Let’s not mince words here; LLMs were designed to make database querying and researching easier by allowing you to make natural language query. The LLM then collates the information from the dataset, but if the dataset is too broad it will produce garbage (hence why chatGPT makes mistakes). That continues to be the use case they are good at. Everything else is a circle being forced into a square.
1 Like
Let’s play a game.
How about I commit to writing multiple content pieces, some of which are AI generated with human editing, some of which are entirely human generated.
Will you be able to accurately sort one from the other without being told which was which before hand? I’m not sure you would.
It depends on how much heavy lifting human editing is doing here.
I can also produce garbage. All humans can. I was reviewing pretty junk technical articles written by paid human freelance authors well before LLMs were a tool to abuse.
If we want talk about quality controls we can have that discussion without referencing any assumed (and unprovable) use of LLMs.
But if we continue to be this biased against the use of LLMs they are just going to get used quietly and sneakily because telling us about how LLMs are being used becomes too risky. A hard no incentivizes the wrong behavior and kills our a ability to influence appropriate uses that end up increasing quality.
That’s the thing; the appropriate uses, in my view, are MAYBE editing with human oversight, possibly generating manpages (based on code comments, essentially an LLM powered doxygen), and perhaps automating tests. Anything more than that is not only inferior content; it’s unethical.
Somewhat unrelated, but this is kind of an unhinged thing for your friend to do, LOL
1 Like
I respectfully disagree. We can’t always detect plagiarism or outright misstatement of facts, but we discourage these in the strongest possible terms. I think the same applies to the use of AI/LLM for content creation. If an author uses AI/LLM to create content, then it is not really their work and they should know that is misleading at best and unethical at worst, in my opinion.
I see engineers using LLM and when I ask them why, they have all kinds of ‘reasons’. Most of those ‘reasons’ come down to lack of ability to do the job. The worst part is that junior engineers who should be improving their skills are instead leaning on AI and not really learning much of anything. I do believe there are places where skilled practitioners can responsibly use tools like AI/LLM. Writing article for publication is not one of them (IMHO)
Note: No AI was used when writing this comment.
2 Likes
This is exactly my problem. In the right hands, LLMs can theoretically be used to augment the abilities of a coder and improve their workflow. But as it stands, people without the necessary experience are relying on LLMs instead of learning, producing worse code (and worse writing for that matter)
My personal view on this post is that the review process of technical documentation would benefit from systematic check-ups on the writing platform. It would be ideal if the WordPress instance we use could implement custom style guides (or similar rulesets).
AI detection models/plugins on WordPress could help alleviate friction arising from undue expectations on reviewers. We shouldn’t put an onus on reviewers to handle AI detection.
When I submit a draft of an article on WordPress, I get feedback from reviewers on the technical accuracy and originality.
Human reviewers are scarce resources.
3 Likes
But the crux is IMHO not whether something is AI generated or not, but whether this is perceived as such.
Given the rumoured amount of spam websites spun using AI, sooner or later, the biggest players are going to have to react to it, and filter it. Maybe that’s already done. So we have the risk of losing relevance by not being careful here.
Then there is the branding risk. If there is too much articles that look like AI, no matter their origin, people will think we are cheap and do not care about that. Most people do not care, but a sizeable portion of the free software community do care, and I think, as a community project, we want to attract more people who care about something than the one that tend to be ambivalent, because the former are more likely to contribute than the latter.
So rather than focusing on precision, we should focus on the impact.
I also want to remind that we have a Policy being discussed on this thread: AI policy in Fedora - WIP
6 Likes
The impact of poor quality content? Sure.. regardless of the tools being used to produce it, absolutely. If you want to add scanners that help make sure content doesn’t read as spam as part of the review process sure. The irony is, they’ll likely end up being AI based scanners moving forward. AI trained to detect AI genereted content…deeply satifyingly ironic.
The perception that AI was used in content is a big part of the problem for me.
Low quality content was produced, got through review and was published. Those are the observable facts. Much of the rest of the discussion is speculation.
Jumping to conclusions that particularly poor content is the fault of someone using LLM shows a significant bias. Based on this discussion in this thread, I’m concerned that anyone who actually admitted to using LLM prior to review in their drafting process as an aid would get a fair review on the merits.
Funny enough, detecting AI is something AI is really good at, so I agree here.
I’m curious to see if the scanners can detect my use of Grammarly, which claims to be LLM-based. I haven’t been completely transparent about using it in my long-form writing up to this point.
It’s difficult to argue that Grammarly’s support reduces quality compared to writing alone. I also doubt its tone suggestions for my sentences would raise any concerns.
For help with writing this response, I asked Grammarly to respond to these AI prompts:
Prompts created by Grammarly
- “Improve it”
- “Make it persuasive”
- “Shorten it”
Something like Grammarly, while I personally don’t use it, I find more acceptable than other LLM editors, largely because it’s real time. It’s an assistant, not something you give your work to and simply ask it to edit wholesale. At least, that was the case last time I checked it out.
The point is.. its not a blanket sea of garbage hype out there. There are islands of utility.
Here is one tool that is probably “not a bad thing”{tm} because its well scoped as an assistent tool when used in some ways.
Google’s notebookLM is probably of a similar category of “not a bad thing”{tm} when used in certain ways when given early draft content as input.
What we need is to build up some understanding of what the reasonable uses of the tooling that lead to better final draft outputs in a multiple draft process. Partly so we don’t have crap output, but partly because we’ll be able to influence and mentor the entire next generation who coming up right now and are immersed in the sea of AI tooling. This stuff is not going away, the best we can do help steer the people towards the best way to make use of it.
this post was not run through grammarly.
WordPress provides a feature similar to what is being describe in Grammarly. It will make suggestions for improvement based on repetitions, sentence length, paragraph length, etc.
I didn’t read all the posts, because I’m lazy but I find it funny and interesting how people hate AI just because. AI are great tools, if you’re not using it, you’re falling behind.
That being said, I’m not in favor of AI generated content, but assisting a writer, it’s not bad. Just a thought, if we could offer an AI to assist writing, instructing that AI with our styling guide, we could prevent people coming with AI content generated elsewhere and we could even probably measure the use of AI.
That was just a wild thought that I had. Sorry if it sounds crazy
There’s no one to blame for the public perception of AI but the AI companies themselves, though. They’ve been shoving it everywhere in places where it doesn’t belong and is actively harmful to people’s welfare. See: AI generated art, slop articles, etc.
That said, I DO agree that there are uses for AI outside of generating things, and I find LLMs interesting in concept. Something like this, if possible, is far more in line with what I think many would consider an acceptable use case. However, it is still an uphill battle. The public perception of AI is damaged, and there’s no one to blame but the venture capitalist firms that overhyped it.
1 Like
I’m not sure that’s true generally with the public.
I think the success of Adobe’s generative AI tooling as an aid for creatives is a counter argument about a more general perception. Most of the people paying for Adobe products are using the AI tooling Adobe is offering them. I don’t see a perception issue when the AI product is actually well scoped and well marketed.
There’s certaintly pushback in a portion of the creative space, acting specifically, because of the employment impact there. But they are sorting out the ethics as part of union contractual obligations. It’s useful to have a union to help navigate and establish reasonable norms. But the tool use will end up being normalized in those fields, its already happening. Read up on “Watch the Skies” movie and how AI is being used with the consent of the actors for the benefit of the audience. This is not an unending sea of garbage, but skeptics with real concerns can’t just bury their heads in the sand and ignore the tools.
The perception in the software engineering community is probably damaged significantly because for a lot of us this technology is being marketed and being implemented by our employers in a way that is a direct threat to our employment. It doesn’t help that this field of work is broadly ununionized and there is no collective bargaining voice for labor to help set ethical norms on how the tools are being used. I never thought I’d be both a technologist and a luddite instantaneously in the same moment but here we are. Welcome everyone to the 3rd or 4th wave(depending on how you count it) of the industrial revolution. I’m sure in the future there will be lovingly handcrafted artisanal software that people will own and display as status symbols, or in art galleries and museums, but that’s not what the general public will be using day to day.
The perception in the open source community is damaged for sure beyond the above reason because of the cavalier attitude taken towards copyright around training data by key companies who are out in front now in the marketplace. it feels like I’m condoning that behavior by touching the tools at all. I’m sure others in this community feel the same way. But like so much in human politics, working outside of the system that allows behavior you object to just ends up meaning you have less and less impact in fixing the bad behaviors.
The magic of copyleft and open source formulation is that instead of standing outside of the copyright system that allowed for behaviors that people disagreed with on deep ethical grounds, they found a way to work within the system and stand up a competing ethos for people to form projects around…and many of them contributed to that competing collection of software using some amount of proprietary software. We’re gonna have to do something similar, as we stand up something more aligned with our shared sense of ethics in the AI space and start putting ethically trained models into service as competitors to the models we have problems with.
If anything we are the way out of the trap. The floss community, because we are a community built around an ethical approach to software and technology, are the only people who can engage and build a competing set of tools and show its advantageous for society to do things in a more ethical manner. if we don’t engage, if we don’t do it. the less ethical way wins. We can’t rely on laws and legislation to solve this problem, we have to promote an ethical way forward…and that requires engaging and using our skepticism not as stop energy but as motivtion to identifying and solving the problems we are seeing.
1 Like
For some use cases, yes. But almost all artists I know are more upset about diffusion models stealing their art and welfare than anything else – most artists (both those I know personally and those I simply see online) either despise AI generative tools entirely for this reason, or if they do use them, use it only for basic stuff like layouting/color correction/etc. Independent artists in particular are struggling to compete with so-called ‘prompt engineers’ and ‘AI artists’; even though the AI art is noticably worse, there’s a faster turnaround time and thus the ‘prompt engineers’ can churn out more art at a cheaper price. Of course, this isn’t exactly sustainable – Midjourney still isn’t actually making money, and is relying on investment funds. Eventually, they’ll have to raise the price, at which point I believe things will go poorly for diffusion models. Still, Diffusion is different from LLMs, as much as they are often conflated, and thus only tangential to this topic.
We’ll have to see where it goes, to be honest. I’ve seen some voice actors taking jobs for their voices to be trained to create voice models, for instance, and while that might take work away from others, it is at the very least consensual. The big issue seems to be, at least among VAs, that companies are taking advantage of their contracts in order to keep using their voices well after they are no longer working with said company. Remember Rogue One? Obviously the technology was different back then, but there was a lot of controversy from Peter Cushing’s friend, claiming they used his voice of a dead man without consent. There is indeed something morbid about that, and I would want to see regulations against that.
If I can speak to this somewhat, it’s not just the fear of replacement. I have a friend who used to work at Meta, specifically with their AI stuff, and their biggest complaint was executives asking for them to find usages for LLMs in places where there were none. They would waste months working on a feature requested by an executive, only for it to inevitably fail; then the exec would say “Well it must just be the implementation, not the technology. Let’s try again!”. That’s something we as the general public don’t see. The products we see are the ones that make it through that filter; countless devs feel like they are wasting their time implementing something with no actual usecase. I’ve heard the words “I’m working on nothing that matters” tossed around, and while I’m sure we can all relate, that’s pretty grim!
Beyond that, I’m less bothered by being replaced and more by the terrible Junior programmers it is creating. I’ve run into programmers half my age who claim to be top level programmers, bragging about how great they are, only to then look at their code and discover it’s all AI generated. Not only that, but it’s all so fragile. Every slight change breaks everything else – this is a problem in software development in general, but it’s even more exacerbated by AI generated code. The LLM is not capable of future proofing.
AI is nice for things like summarizing git commits, or generating documentation, but so many junior coders are not learning and instead just relying on LLMs. It’s like the problem we’ve had for ages with, say, Stack Overflow, but even worse. At least with Stack Overflow, people would eventually have to step out of their comfort zone and write something and fail on their own and learn. Now, they can just ask an AI to spit out code that works… for now. In short, I dislike AI in tech less because of a fear of replacement, and more because I find vibe coders to be insufferable incompetents who make everyone else’s jobs harder.
I agree. I’ve said there are plenty of usecases for AI – and I do mean that, genuinely. I just do not think writing articles for Fedora Magazine – or any news source for that matter – is one of them.
1 Like