Ok, I’ll reframe the whole thing and get into more detail hence:
How would a proper filter look like in modern times? I’ll provide an example + a breakdown of the major issue:
Example start
A basic filter would involve:
- Building a list of applicable profanities
- Developing a method of dealing with derivations of profanities
A moderately complex filer would involve, (In addition to a basic filter):
- Using complex pattern matching to deal with extended derivations (using advanced regex)
- Dealing with Leetspeak (l33t)
- Dealing with false positives
A complex filter would involve a number of the following (In addition to a moderate filter):
- Whitelists and blacklists
- Naive bayesian inference filtering of phrases/terms
- Soundex functions (where a word sounds like another)
- Levenshtein distance
- Stemming
- Human moderators to help guide a filtering engine to learn by example or where matches aren’t accurate enough without guidance (a self/continually-improving system)
- Perhaps some form of AI engine
Example end
As we can see it’s 3 major categories (btw that’s a modelling example copied from a older posts of someone working professionally as a coder for profanity filters)
They are ‘Simple filter’ ‘Moderate filter’ and ‘Complex filter’.
A basic filter does a great job and catches a good 70% of profanities. This is the part which causes the least interference with general communication. That is fine. Absolutely for it, works, proven, eases workload substantially.
Good to implement!
Then we have the moderate filter, that one becomes already iffy, the pattern matching is easy to cause problems, l33t is a given to be taken into account and I would say should nowadays fall into ‘basic’ actually.
False positives? Well… there’s the crux of the problem. How to ensure that? You can’t. You can’t even ensure that it’s a positive properly, hence going a step further to ensure it’s a false positive if we can’t even decide if it was a positive in the first place is already a major issue. Here the filters start to break into the area where it becomes bothersome and nonsensical already, able to cause issues.
Complex filters… we don’t need to speak about that. White- and Blacklists should generally be at the basic area again, I don’t know why it isn’t, that’s a given, using a dictionary to implement those should be standard practice anyway.
For the others? bayesian interference is a mess which can cause a ridiculous amount of false positives, soundex can cause even more issue, Levenshtein distance is decent but also… makes it so when you try to form your sentence different when nothing is seemingly ‘odd’ in it… that sentence will still get blocked while it shouldn’t have been in the first place. Stemming we don’t even need to think about how that can cause issues… what if the derived word has a completely different inferred meaning from the stem? Oof, that is a false positive machine basically.
And AI is not advanced enough to handle it either, deriving context is nigh impossible for it still, with simple situations it can help, with anything beyond the simplest things though it causes a mess.
And that was roughly 8 years ago in terms of complexity for profanity filters.
Which leads me to this:
Yes, and that’s all that’s needed to stop a large portion of people to go further with it. That singular step to have them re-think.
Aligns with ‘simple filter’ and needs nothing beyond.
Now towards why am I saying ‘it’s more harm then good’ (The next major point beyond this is the direct links and explanations of concepts):
Generally profanity filters are unable - inherently - to filter out 100% of it, you can insult, harass, defame and so on… every in a myriad of creative or simple ways which removing it fully would utterly neuter communication completely as context is nigh impossible to infer with a automated system, even nowadays.
That means beyond the ~70% capture rate you increase the rate of false positives substantially more then you reduce the actual amount of proper positives. That means the efficiency ratio of the filter goes by design down… maybe won’t in a few years but currently that’s still the status quo.
Next up is that a profanity filter has to be adjusted to the respective product’s userbase. In our case this is actively stated in the LE ToS under Paragraph 10 III ‘Account Creation’: You are over eighteen (18) years of age;
This deems that terms which wouldn’t be fine to use in any environment with minors instead can be used freely. Hence for example ‘sex’ is to be a freely used term, unless it directly relates to depiction of those acts rather then the overall maturity topic of it. This term as well as any other possible terms which could come up in a discussion between informed adult people without depicting it and hence going over the line… which is a blurry one definitely, major issue for overall moderation in general.
The same goes for terms describing ethnicity groups for example, or physical traits people can have.
As a end-result the filter is only allowed to actively weed out distinct words which are in basically 99% of situations used as… well… a profanity. Anything beyond is limiting general communication. Specific things like religious or political topics can also fall under that, though it needs to be done properly and through the board, otherwise we would get favoritism and that’s a big big ‘oof’ again.
Next up, non-clear language in the rules:
As the Rules of Conduct for ‘Harassment, defemanation, and abusive language’ describe it: While using the forums and in-game chat, all users must treat others with respect. People have the right to participate in the discussions which take place here without receiving abuse or harassment of any kind. This also includes Bullying, Name calling, Baiting, and slander and will not be tolerated in a public or private fashion.
Well then… what exactly is ‘Abuse’ or ‘Harassment’, in that case specifically as mentioned ‘Bullying’, ‘Name Calling’, ‘Baiting’ and ‘Slander’?
By which law? Is it our respective law of the country of the user, is it the US since EHG has their seat in the US?
For example the US has so called ‘Freedom of speech’ while my country has ‘Freedom of opinion’, quite different terms with different meanings, importantly so.
This for example depicts that in my country nobody is allowed to stop me from voicing any form of opinion in any way… but! If it’s voiced in distinct ways (Harassment, defamation and so on) then I’ll have to handle the consequences of doing so, but nobody is theoretically allowed to hold my mouth shut for trying to speak out words. So… while I’m acting against the rules of conduct and my message will be removed henceforth… the voicing of my message itself is not to be hindered, it’s to be punished afterwards.
Quite important aspect.
Now, let’s move over to why it’s harmful directly, supported by studies:
Conflict tends to get less when expression is unhindered, this has been proven scientifically already. The more freedom of expression is there the more likely it is that conflict won’t happen as the ability to solve it before escalation is provided.
https://www.ifn.se/media/dvsnxius/wp1473.pdf <— this is a study done on how freedom of expression and social conflict are correlated to each other.
This means that inhibiting the ability for free expression, hence the usage of distinct words, not how they’re used has commonly a negative effect on a community rather then a positive one. Depicting that measures are taken they have to be as limited as possible.
Next up is the distinction of what counts as ‘harassment’ for example. That commonly is depicted as ‘The intentional annoyance, threatening or demeaning of a victim’. Hence, distinction is to be provided for un-intentional behavior, hence compulsive behavior as well as uninformed actions which leads to a unwanted outcome.
This can’t be handled through a qualitative profanity filter, it has no meaning in that regard. The biggest aspect to hinder there is the pure impulse of doing it, not letting it come to realization, which causes any complex filtering and limitations to have no effect in the first place. Hence beyond the basic premise solely reducing the ability for ‘normal’ communication.
Defamation is also a problem, that one can’t be handled with a filter in the first place. It’s the spread of false information about a person, business or organization. Libel and slander hence. How would a filter decide if information is false or true? It can’t, unless the operators of the program implement those filters… which is a clash of interests by design.
So also a non-viable approach.
Bullying also can’t be handled with a filter, since it’s ‘repeated and intentional use of words and actions against someone or a group to cause distress and risk to their well-being’. Hence it is repeated harassment, already covered.
Name calling… oof, that’s a tough one. ‘A form of argument in which insulting or demeaning labels are directed at an individual or group’.
Well, what’s ‘insulting’? And what’s ‘Demeaning’?
Insulting would be ‘an expression, statement or behavior that is deliberately disrespectful, offensive, scornful, or derogatory towards and individual or a group’. Aha! Once again, intent has to be there. So a ‘insulting’ term can only be one when it’s used intentionally so, otherwise it is not insulting, no matter the word.
Now we start to get into major problem areas already.
Demeaning on the other hand is ‘behavior or speech that is belittling, insulting or derogatory towards someone or something’. Well, we covered insults already, so ‘belittling’ is ‘the intentional act of making another feel worthless, empty and dismissed’. Once again, intention.
So that leaves us with the term ‘derogatory’ as the last one ‘a word or grammatical form expressing a negative or disrespectful connotation, a low opinion or a lack of respect towards someone or something. It is also used to express criticism, hostility or disregard.’
The bold part is an issue.
How to make the distinction? Well, that’s not clear, which is why it’s a problem. It’s the only aspect which works without intent, but intent matters. That’s why it’s so often used, because it just can be done ‘whenever’. Also everything can be derogatory depending on circumstance, which means everything is derogatory by default. Someone messes something up and you say ‘Well done!’ then that term has become derogatory, it’s a disrespectful connotation.
But… should decisions be made on how much you’re respected? No, they shouldn’t. Respect is not a inherent thing you’re getting, respect has to be earned through actions, which also means the negative form of that can be provided through actions. Should you voice it though? It might become important, most of the time though ‘no’.
So yes, it’s generally detrimental as automated filters can’t take ‘intent’ into consideration, we can only infer on ‘likely intent’, which has to suffice as anything beyond would cause negative effects of labelling everyone automatically as ‘ill intended’ by design.
Never said that. As a victim you’ve nonetheless got to provide the information that ‘you’re a victim now’.
If others get to decide beforehand if you’re a victim or not that takes all the responsibility and authority out of your hands, it doesn’t ‘empower’ you as it should, it ‘depowers’ you.
That’s very important for someone who’s gone through a traumatizing experience, they often feel powerless, so don’t play into that further and cement it through your actions.
Never said that either.
Carebearing is being overzealous. Which yes, EHGs filter is. Don’t you agree? I think you agreed earlier, didn’t you? So why not anymore?
Also I never spoke ill about someone or called them ‘snowflake’. The problem arises if you try to save 1 person and instead throw 1000 under the bus at the same time. That’s not how it works, that’s detrimental.
You gotta use appropriate measures hence to ensure this doesn’t happen.
That’s all I’m saying here and always did. Profanity filters are a necessary evil as we don’t have a better system created - yet - but hence since they’re nonetheless a evil they need to be used as sparingly as possible to not infringe on fundamental societal aspects.
Neither/Nor, first of all, thanks for providing a prime example of ‘derogatory’ wording, it’s a really good example.
Secondly, I’m advocating for proper measures being provided. You can’t run out and scream at any problem a meme like ‘But think of the children!’… because that’s not how it goes.
Yes, all the aforementioned issues are… issues, major ones, societal ones, systemic ones mostly. And you need to provide according options to ensure that when something happens it is handled appropriately. You cannot avoid it happening in the first place, it’s a current impossibility with our knowledge base world-wide still, maybe the future changes that, hopefully so. Hence… you need to adjust to that circumstance, as shitty as it is.
Also, do you think the majority of those cases are in any way/shape or form ‘easy’ to handle?
Was it intentional? Has the ‘victim’ in such a case actively gone into the situation with knowledge that it’ll happen? (A form of enabling btw. as little as we like to see it. If you wear a shirt stating ‘punch me’ while running around screaming to everyone to punch you… you shouldn’t be surprised to be punched after all when it actually happens) Is it ‘solely’ a clash between cultural differences leading to it? What sort of infringement has actually be done?
You have to take those things into consideration individually, that’s why it’s so darn hard to handle those things.
And also There’s often 2 perpetrators and victims at the same time. Obviously not every time… but we’re not talking about rape-victims here… and even in that case there’s so much baffling stuff going on there that it’s a mess.
Such situations are never simple. And it’s definitely not in the hallmark of a game-dev to have the capacity to handle them in the first place. That’s to be done by legal authorities and therapists respectively.
The devs solely should report it.
Now we could talk about how effective those reports often are, but that’s another topic in itself and a mess in many areas of the world currently.
And it would be a good change I would argue.
Having a simple ‘contact list’ and a ‘friends list’ separately goes a long long way, especially in handling harassment as for a contact you wouldn’t inherently see if they’re online or not, just having the ‘quick dial’ button basically available.
Luckily not my place, but yes… it is.