Square Enix actually has a pretty sick automated QA already. There’s a cool talk about how they did that for FFVII remake in GDC vault, and I highly recommend watching it, if you’re at all interested in QA.
It has nothing to do with AI, it’s just plain old automation, but they solve most of the issues you get with making automated tests in non-discrete 3D playspace and they do that in a pretty solid way. It’s definitely something I’d love to have implemented in the games I’m working on, as someone who worked in QA and now works in development. Being able to have mostly reliable way how to smoke-test levels for basic gameplay without having to torture QA to run the test-case again is good, and allows QA to focus on something else - but the tools also need oversight, so it’s not really a job lost. In summary - I think the talk is cool tech and worth the watch.
However, I don’t think AI will help in this regard, and something as unreliable and random as AI models are not a good fit for this job. You want to have deterministic testcases that you can quanitfy, and if something doesn’t match have an actual human to look at why. AI also probably won’t be able to find clever corner-cases and bugs that need human ingenuity.
Fuck AI, I kind of hope this is just a marketing talk and they are actually just improving the (deterministic) tools they already have (which actually are AI by definition, since they also do level exploration on top of recorded inputs), and they are calling it an “AI” to satisfy investors/management without actually slapping a glorified chat-bot into the tech for no reason.
That’s a stupid idea. You’re not supposed to QA or debug games. You just release it, customers report bugs, and then you promise to fix the bugs in the next patch (but don’t).
No better testing than in production.
Or do the Bethesda thing and let people playtest their slop and fix it for free.
Not buying their next game it will be a nightmare
I’m cautious but a little curious about this one, because QA could actually be a very good target for AIs to work with.
- It might not kill jobs. Right now, engineers finish a task and the limited number of QA engineers can’t possibly test it enough before release. That game-breaking bug you found in a game? I’m sure some QA had it in their plan to test every level for those bugs, and yet they just didn’t have enough time - and the studio couldn’t justify hiring 20 more QA squads. Even if they do upscale AI testing, they’ll need knowledgable QA workers to guide them.
- This is often extremely rote, repetitive work. It’s exactly the type of work The Oatmeal said is great for AIs. One person is tuning the balance on the Ether Drive attack, and gives it an extra 40% blarf damage. He tries it, sees it works fine, and eagerly skips past the part of the test plan to verify that all cutscenes are working and unaffected to push it in. An AI will try it out, and find: Actually, since an NPC uses an Ether Drive in a late-game cutscene, this breaks the whole game!
- Even going past existing plans, QA can likely find MORE work for AIs to do that they normally wouldn’t bother with. Think about the current complexity of game dev that leads to the current trope of releasing games half-finished to eventually get patched. It won’t help patch games, but it’ll at least help give devs an up-to-date list of issues.
That said, those talking about human creativity and player expectations are still correct. An AI can report a problem with feedback that a human can say “No, that looks fine. Override that report.” It will also be good to do occasional manual tests, and lament “How did the AI think this was okay??”
Remember when square used to make great games?
That was Squaresoft, not Square Enix.
My memory failed me. Thanks for the correction.
Square Ewwwwnix
I loved Enix’s Ogre Battle and Square’s Final Fantasy 6 and 7. How could putting the companies together make a bad?!
Chrono Trigger remake, first all AI development, because we live in the bad timeline.
spits
Was gonna make a post with this article but this is related so I’ll drop it here instead.
“Square Enix is laying off more developers in the UK and US as it refocuses on Japan The publisher has expressed interest in replacing development roles with AI”
70% by what metric?
Is that going by bugs identified, fixes implemented, headcount?
A lot of hate in the comments but IMO this is one of the few things that LLMs are actually really good for. It’s a shit job nobody wants to do that LLMs are really good at. Notice that they said 70% and not 100%. Yeah that means they’re probably going to have 30 people doing the work that 100 people used to do but people are still in the picture overseeing things. Automation isn’t, by itself, bad. The bad part is that our whole society is built on the idea that your entire value as a person is based on being able to work and make money and job loss is way worse than it should be.
I’m just shocked any publisher is still doing QA at all, with the state AAA games release in these days.
That’s why they want 70% QA from AI. Because right now their games are only 10% QA tested.
30% of the the QA team will be working at 300% capacity - can you guess which 30%
And I thought I had no more disappointment left to allocate
It’s just that you’ve reached your free quota, further disappointments will be charged 0.0937 emotional stability per hour
Inb4 their games come out even more broken
Oh. Gross.
Some AI or central computer going haywire and destroying everything is, like, the third or fourth stock RPG trope just behind the Dark Lord burning down the protagonist’s village in the first act or the mysterious waif girl actually turning out to be a princess.
You really think they’d know better.
In my country we say ajajajajajjjj
