I think it’s great that you want to engage your playerbase and get their feedback on important issues, but unfortunately the method you are using is not representative of the whole. The method of posting surveys and polls on Discord or the forums leads to self-selection bias, and in quite a strong way.
To look at the latest one, we have multiple layers of issues:
-
Only those that read the forums/reddit frequently even knew about it. Yes, it was also linked from steam, but that’s no different from the forums. Majority of players never come on to the official forums, or only do so for things like patch notes. I’m sure you can compare your sales to forum activity to verify.
-
Those that were aware had to opt-in to the survey, meaning you generally are only getting those with very strong opinions on the topic. That means even within the subset of the forum going population you’re only getting those that self-selected into it.
-
It was a google form, so again lowering the number to those that use google accounts.
The best way to handle these is in game, with short single question polls. Randomly select players (or the first X% of players that login) get the question before they get to the character select screen. It wouldn’t be every player, but it would be a random sample which is a good measure of the actual player base. A short question isn’t a major burden. Track to ensure no person gets questioned twice.
This also ensures that those voting are actually players, and not just trolls. To further improve the quality of the results you can have multiple variants of the question. This minimizes bias that may come from how you worded the question.
That right there would give you a much better picture of the totality of the player base, but you could go further if you wanted to get a more accurate picture. As they are doing it from the game client you can capture demographic data about the user (hours played, number of characters in league, primarily online/offline, etc). Using that with their answer you could do some basic regression analysis to get an accurate prediction of how every user would vote if asked.