So with 1.0, we came across a lot of issues with new - falconer and warlock - and newish - runemaster - masteries. Mostly, but not only related to Ward, but there were other issues.
It’s not possible to catch every problem, but there was definitely a theme surrounding what went wrong with 1.0. From the outside, it would appear that some of that stuff might have been caught given sufficient testing.
Last Epoch was an Early Access game for a long time. It is not in EA anymore. It’s in production, but it seems the developers are still treating the game like it’s in EA. Now that Last Epoch has been released, maybe they shouldn’t treat players who bought a released game like testers anymore? Maybe they should invest in some testing infrastructure?
I have no idea what the size of their testing team is, but I don’t think it would have made a difference. You had the same thing happen in D4 and they definitely have a HUGE testing team.
There are many things that are only caught when testing in large scale. And some things only happen because a few crafty people come up with ways to interact with the game mechanics in a way to break them, and then word spreads. So unless they’re a part of the testing team itself, it will keep happening. Like it happens on all games, PoE included.
That’s not what happens when a game launches, and that’s not how they treat us players.
Just because a game launched, it doesn’t shut the door for bug reporting. On the contrary, good games incentivize their players to engage and speak up whenever bugs are found.
And if they treated the game like EA and us players like testers, they wouldn’t make polls asking what we want them to do… they would just do things and let us test when it’s released, you know, like they used to do when the game was in EA.
It isn’t only about the size of the team, it is about the quality of testing procedures and documentation.
Reliable, planned, and documented tests that can be repeated.
Example:
You design a node that is supposed to increase the area of a skill based on Dex - You create an example character with defined stats, and formulate an expectation about how large the skill should be at x,y, and z Dex, then you run the test with the values for X,Y, and Z and check if reality meets expectation.
I’m a programmer. I know how testing works
I also know that there is never enough time to create case tests for every single thing. Which is why bugs happen.
Furthermore, many of the issues arising in these games are because someone goes “What happens if I decide to stack X and make skill Y work with whatever?” and sometimes they break the game. More often than not, it’s something that the devs never even thought of.
Consider this an open reply that is written so that people without experience in software development can understand my points. No insult towards you intended by over-explaining things you probably already know.
Yeah, it is not feasible to test every possible interaction in a game as complex.
But the more complex a software can be expected to be, the more thought should be put in designing a good process and environment for tests. Saves a lot of pain eventually.
Designing a test case for every node of a skill, only testing the directly obvious interaction, is something I would establish as a process. A skill changes the damage type - check if the damage type is really changed by reading debug information directly from the system. It says a skill tag is changed or added? Check if the change happened. Have a visual check to confirm that it is reflected in the node’s description.
Coming from SIL4 train automation, I’m a bit biased towards extra caution, though I know that the same level of caution is financially unrealistic for an indie computer game. People can live with a bugged skill, but not with bugged brakes on a train
Yes, this is the main issue. We have a small-ish team and we always have a lot more things to work on that time to do it, so we have to establish priorities. In out case, we design tests for things regarding the billing and financial aspects of the software, because those are the ones that impact the clients the most.
We would like to have more tests, but there simply isn’t time. Sometimes we get to expand on it a bit when we get a new employee (sometimes an intern, sometimes a new hire) while they’re getting to know things, but overall there are lots of things that can’t really be tested internally due to that lack of time.
It would be easier if we had a testing department, but we’re not big enough for that.
And the most important aspect of all this that I want to point out is that when you have a very complex system where many parts interact with each other, making case tests for everything becomes exponential. For example, you start with skill A and add case tests for it (we’re going to ignore all the other mechanics for simplicity). Then you make skill B and now you have to make test cases for it and for A+B. Then you make skill C and now you have to make test cases for A+C, B+C and A+B+C. And so on.
Eventually you reach a point where you spend more time on the case tests than on the skill itself. So a balance must be reached where you test some things but hope the rest get caught up in the testing department.
There’s a reason why even Blizzard, who is so prone to simply dump money on everything, has the same issues. They also need data from a big number of players to actually balance things and even then they still fail. It’s not an easy issue.
Yes, you need a different approach for things that can kill/maim people. Much like banking software also needs this. The tolerance (and more importantly, the consequences) for failing to do so are much more drastic.
Well, I think a bigger frustration is having testing, but not fixing bugs reported during that testing. ala, Blizzard open-betas. From what I hear, the ward issue with warlock was reported during IT, but still made it through to release.
I don’t know about it being reported or not. I haven’t heard anything about it, but I don’t tend to follow things like that anwyay.
Yes, I feel like bug fixing should have a higher priority. At least the most important ones that impact gameplay. But I can also understand that it’s complicated when you have a list with hundreds of tasks to implement and sometimes bugs get pushed back in favor of new things. It’s not an easy balance.
Yes, the procedures they use definitely need work still. They’re a fairly new company (put into perspective with other companies of the genre) and hence have a bit of leeway. It’s simply one of the things which clearly haven’t been ‘optimized’ yet to the extend players would like.
To be fair, some of them sometimes aren’t bugs, they’re just interactions that the devs didn’t anticipate and make things go boom.
This happened in Scourge league in PoE where one of the new currencies allowed you to go into many MANY negative resistance (way below the -60) and through interaction with an item (I forget which one) effectively made you immortal. They even had to make an emergency patch about it and clear those results from the leaderboards barely less than a week in.
If you have not formulated an expectation/requirement, you won’t test against it, usually.
But when you design a skill and expect a specific behaviour, you can test if your implementation meets the expectation.
This doesn’t account for balancing issues. Your skill can behave as expected and turn out to be unbalanced. The problem arises when formulated expectations don’t match reality - that’s a bug. Everything else is a feature
The playerbase will always find interactions that are stronger than anticipated. It’s a coordinated brute force attack against your design. They inevitably win by numbers.
I would settle for actual bugs being discovered, and those discoveries acted on pre-release. The bug-finding wouldn’t be of much use unless there was corresponding bug-fixing.
In all cases that have been controversial the changes have been bug fixes, and not unintended interactions.
Yeah, that’s my main gripe. I understand everything being said around here by our programmers.
There isnt enough resources to catch all them before release, but some bugs and broken interactions were so obvious that it’s hard to believe it wasnt noticed during test phases.
And I agree with OP. Needs to have investment in testing and balancing skills/ interactions. Specially now it’s not EA anymore.
Seems like community testers alone isn’t enough. Or are they hiding some of the broken things to have a head start?
Or are they reporting the things but Devs aren’t taking action?
It never is, not without reason the technical part of releases for PoE have risen massively since they’ve implemented their own personalized testing environment a few years ago.
Before: bug galore
Now: sometimes bug
But things like ‘the value of a node is multiplied by 10 in outcome’ wouldn’t be possible that way. That’s ‘we didn’t test it’ which is a viable gripe to give EHG a big negative point.
Why? Because that’s the basic things which have to be tested.
Oversights like this are the same quality as Wolcen presented during their release, with a building leading 100% of the time to a save-crash until it was fixed.
Those types of bugs are not excusable simply.
So it’s something which is a mandatory part for them to do in the future to avoid similarly serious repercussions, they’ll have enough headaches from mechanical interactions which weren’t ever expected, the should at least have a grasp on very very clear-cut expected ones.
