"Ideas like this usually just die": Inside generative AI’s effect on modding, and how host sites struggle to maintain transparency

I’ve been scrolling through Nexus Mods near enough every weekday morning for three or four years now. On a good day, I might spot a couple of new mods potentially worth writing news about and a couple, usually Skyrim player homes, I might fancy checking out in my free time.

Historically, browsing for mods was largely just a matter of gauging which looked complex or unique enough to be worth investigating further, and making sure nothing about them appeared offensive or otherwise dodgy. Usually, new quests, new mechanics, or substantial overhauls which can turn a game on its head got the nod. Then, a few years ago, AI-generated voices started popping up in the former. Skyrim, being the most actively modded game, was ground zero.

AI voice tech called Elevenlabs was used to either impersonate the base version’s voice actors or generate lines for new characters. Nexus Mods, seeing these works emerge, put out a statement in 2023, outlining that AI-generated mod content wasn’t against the site’s rules, but "may be removed if we receive a credible complaint from an affected creator/rights holder".

Three years on, and with generative AI being more prevalently pushed than ever - regardless of the global memory shortage and computing crisis it’s caused - the use of it in mod-making has only grown. The generated voices are still about, but are now joined by AI-produced art and, most recently, the use of LLMs to put together the mods themselves. The default option for folks like me, who prefer their mods human-made, or to have the option of a version that doesn’t rely on AI, is to ignore mods that use any of the above. However, as AI use continues to grow and diverge, it’s only getting tougher to recognise which do or don’t rely on these tools.

The other week, I stumbled across a Skyrim mod called Bard Hero. The premise of it is to turn the ability for your character to play instruments, which has been added to the game by previous bard-centric mods, and turn it into a full Guitar Hero-style rhythm game. An impressively ambitious addition, and the sort of thing I’d typically give further attention. In a lengthy description, though, its creator - a modder going by Mern - disclosed that they’d "used LLMs to help for coding, debugging, documentation, and the occasional 4 am ‘huh?’ moment".

"I will mention I have a diploma in programming and worked at a game studio," their disclosure continued. "I had to think architecturally when designing the mod; I built and tested it, reviewed and adapted the code, and maintain it. The source code is open source if you want to take a look. AI use is something people have strong feelings about, and rightfully so. With questionable AI practices and grandma’s pension ending up in the next AI IPO, there are plenty of reasons to be cautious. We are, after all, apparently speedrunning technofeudalism."

Since spotting the mod, I’ve reached out to Mern, aiming to learn a bit more about why he turned to AI, given his acknowledgement of its controversies and his past in game development. Bard Hero, it turns out, isn’t the first time Mern’s used generative AI in mod development, having dabbled with AI voice generation app xVASynth and used AI tools to make a follower named Umbrael. Between then and now, he’s made other mods "with no AI involved, using Papyrus, ESP, hand crafted animation work and learning Skyrim Script Extender (SKSE) development". Recently, he’s turned his modding attention towards "using LLMs to help develop SKSE plugins".

"I've always preferred SKSE implementation, because it allows you to do so much more," he told me. "SKSE development requires working with C++, but I mostly studied and coded in C#. C++, being a lower level language, is much more difficult to develop with and I didn't want to spend too much time learning C++ (I'm still actively in the process of learning it however) and doing tedious tasks such as reverse engineering memory addresses that aren't supported by CommonLib or AddressLibrary. I also just generally wanted to see how LLMs would perform in such a different environment. From there, I used LLMs and worked on Live Audio Output Switching SE. Then I wanted to see what else was possible, becoming more ambitious and working on projects like Apparel Preview, Fitting Room, Menu Studio and eventually Bard Hero."

In terms of why he chose to use such tools to develop Bard Hero specifically, he explained that the idea for the mod came to him in the shower. "My girlfriend had bought a Wii guitar controller and I wanted to know whether I could get it working in Skyrim,” Mern recalled. “Mad ideas like this usually just die, as you're not just going to dedicate a year to something that might not even be possible.

"And Skyrim being, well... Skyrim, quick prototyping matters more than usual. You'll hit engine limits constantly, and most of the work is finding a creative way around them, so being able to try five approaches and go with the one that could maybe actually work is worth a lot. The other half was FLICK, a UI framework my friend Fuzzles was building. He talked me out of pure ImGUI while we were discussing Fitting Room's development, and I was impressed, and it was the right foundation for something this mad.

"If I'd opted out of using AI tools, it would have taken me significantly more time to create Bard Hero and I'd have to learn and develop other skills I do not currently possess," he continued. "While technically possible with my background, it would have taken months, maybe a year or more, of very hard work. Now, would I actually be willing to dedicate that much time to such a project? No. LLMs let you prototype fast enough to answer questions like ‘Sure... it's a cool idea, but is this even possible?’ in one evening instead of a month. So, I ran tests to see if the project was even viable, like could I read a Clone Hero chart or could I keep audio locked to charts within a millisecond? If I couldn't do that, I'd have lost a few evenings instead of half a year."

Of the previous bard mods that Bard Hero builds on, the most notable is Skyrim's Got Talent, by prolific Skyrim modder JaySerpa. As of writing it’s amassed 9.5 million total downloads on the Nexus, and wasn’t built using generative AI tools. Its creator has put out a number of mods which add new voice lines, but to my knowledge has always opted for either hiring real voice actors or splicing together existing dialogue lines. Back in 2023, he acknowledged the arrival of AI-generated voices in Skyrim modding, but made clear he felt "conflicted about it".

"I have no plans to clone any vanilla voices at the moment," JaySerpa wrote at the time. "Putting two lines together from within the game files is quite different than training an AI network with another person's voice and having them say whatever you want. Voice cloning is probably going to get some pushback from the voice actors we know and love (it's already happening) and will eventually be regulated, so even though I'm impressed, I'd rather take a cautious stance for now."

I was surprised, then, to find that JaySerpa had been credited as helping Mern to playtest Bard Hero. While he politely declined my request for his current views on AI mod-making, he did explain the sense of modder solidarity that prompted him to help out Mern. "I never close my door to any fellow mod author that comes knocking, so that's what went into my decision," he said. "I haven't been involved in the creation of Bard Hero beyond being aware of it, granting permissions and testing it briefly. I generally always grant permissions to anyone that comes asking, so this was no different."

I also asked Mern whether the pair, who’ve known each other for a while, discussed the use of AI in Bard Hero’s development when having this chat about permission. "We talked about it, and I told him about the indirect modification Bard Hero would make to Skyrim's Got Talent," the modder told me. "I was upfront about using LLMs, and he gave me permission while advising me to publish the source, which I already do for all my SKSE projects, so we were on the same page."

Right now, modder-authored AI disclosures - in the form of both Steam-style descriptions, and published source code - are key to keeping potential downloaders abreast of what’s gone into the mods they’re playing. On Nexus Mods specifically, as with adult content, tagging is the site’s main method of helping users sift through and find AI-free mods. On July 29th, Nexus announced they’d split the single AI-Generated Content tag they’d had for a while into three more specific labels: AI-Generated Content, AI Media, and AI Assisted. The aim is to offer searchers a more detailed picture of what exactly AI has been used for in a mod’s production

This meant Mern had to re-examine how he’d tagged Bard Hero. "[I] decided AI Assisted didn't honestly fit, because the model wrote most of the implementation against my specs," he recalled. "So it keeps the AI Generated Content tag, and I rewrote the disclosure to be specific about what that means."

While Nexus Mods declined to provide further comment on how they’re handling the use of AI, they did point me towards search data showing that of the 180,181 mods uploaded to the site within the past year as of publishing, 6,654 currently have one or more of the new genAI tags. 6,052 of those, unsurprisingly given it was previously the catch-all tag, are solely labelled AI-Generated Content. 258 have been designated as only AI Assisted thus far, while 685 only contain AI Media. Those numbers only indicate what’s been tagged, though. For instance, this Halo: Campaign Evolved mod that acknowledges use of OpenAI Codex as having provided "assistance developing the extraction, matching, validation, Wwise-bank patching, and packaging pipeline" in its description hasn’t been tagged with any of the three AI-related tags as of writing. It won’t be alone.

As is often the case, when trying to determine if a game, mod, image, or video contains undisclosed AI, lines are easily and often blurred. Modders like Mern have their own interpretations of what’s acceptable or ideal in terms of genAI use. In this instance, while declaring it’s just a tool that "can be used well or poorly" and "multiplies whatever the person wielding it brings", the modder isn’t in favour of AI being employed across the board.

"For code, I think LLMs are legitimate, but you should disclose it, publish the source, and test it thoroughly because your name and reputation is on it," he said. "That's most of what I use them for. I'm averse to image generation in finished games and mods, with texture upscaling and placeholders as the exception. I also don't enjoy generated music, and I'll admit that part's personal since I make music myself. Voice is where I'm softer than a lot of the community. I've played around with tools like xVASynth, but I understand why voice can be an issue because there's a real person behind the models."

A modder’s intent, alas, forms only one condition of making sure the mod-tagging system works properly. In addition to the creator putting a similar level of thought into their genAI use as Mern has, they also need to be aware of Nexus Mods’ new tagging rules, and make sure the apply those rules to their mods correctly (or refer themselves to a moderator for help in doing so). Then, those of us downloading mods have to be aware of what exactly the tags denote, check they match an eye test of the description and any visual showcases, and decide where the mod lies in terms of what we personally view as acceptable.

The waters are often muddy enough with games themselves, whose professional developers have a duty to consumers to provide accurate AI disclosures or potentially risk putting their business in reputational and financial danger. With modding, which has always had the blessing and curse of being a relative Wild West, those who don’t play by the tagging rules risk…action from moderators against their account. That isn’t nothing, but it’s not as serious a deterrent.

You’ll not be surprised to learn that I don’t anticipate my morning searches getting simpler any time soon.

Newspaper

"Ideas like this usually just die": Inside generative AI’s effect on modding, and how host sites struggle to maintain transparency

Read this next