[Moderation] Dealing with subtle spammers placing "tiny little ads"

[TRS-80]

This will be the Moderator counterpart to the Announcement thread I made about the same issue where I asked for help from regular forum Members. In this thread, I will discuss methodology I developed to deal with these sort of spammers, once someone has Flagged such post(s).

 

Humor

 

But first, some humor. Does anyone remember this guy? Anyway, that old infomercial kept playing over and over in my head today while I was working on this. I think we stumbled across some of his students! OK, carrying on...

 

What Not To Do

 

First thing to probably point out, let's not simply edit out these links one by one as we come across them. That doesn't really solve the problem. In fact it makes it worse by eliminating the "trail of bread crumbs" so to speak, the easiest way we have to "get them all" (which is by recursive searching). Allow me to explain...

 

These links seem to be posted in groups, by a small number of spammers. Once you catch on to one part of a cluster of them, it is actually pretty easy to methodically work your way through and find quite a few more, and "get them all" (within a given cluster).

 

Modus Operandi

 

But let me back up even more, for those who are totally unaware of this issue. What these people (we think they are human) are doing, is to register, and then post some at least seemingly relevant information in some topic, at least enough to get their post(s) approved. Then they come back some days or even weeks (!) later, and edit their post(s) to insert spam links. To see some actual examples, have a look at the Announcement post linked in the beginning.

 

1. Check Profile

 

So, once I found one such post / Member, the first thing I would do is look at their profile. Typically they will have from 1-5 posts or so. I would right click, and open each new post in another tab. Sometimes you will have to search then in the thread (using your browser search function, Ctrl-F in Firefox) to find the post. You will quickly be able to determine whether they are a spammer or not (you will see all the spam links in all their few handful of posts).

 

2. Flag as Spammer

 

Once you are certain they are a spammer, there are a couple ways you can "Flag as Spammer." Since you already have their profile open, you can click that button in there. Other way is to mouse over their avatar, and then the gear at bottom right opens another little menu that gives you the same option. This should be the first real action you take, because it does a couple things all at once (which will save you some time):

• it Hides (not Deletes) all their posts, and
• Bans them from the site (permanently)

 

Note that the above are subject to Admin settings, and these are the current settings as of this writing.

 

3. Being Thorough

 

Now, if you like to be thorough (I do) I will also kind of go through the threads that contain their spam posts (especially after their post(s)) and look for the following:

• did any other user (usually unintentionally, or on accident) quote the spam links?
  • if so, edit out those spam links by hand and leave a note what you did
• make sure no actually useful content was hidden (I have yet to find this in even one case)

 

4. What Must Be Done (at Minimum)

 

However at a very minimum, you should be taking down a list of all these spam links you come across. This is key to the whole process. Methodically complete all of the above for each spammer and each of his posts. When you are done you should have a list of spam links, and one or more Banned spammers, with all their posts Hidden.

 

5. Working our URL List of Spam Links

 

Next step, we do a regular search on the forums for each of the spam links in turn. I like to leave this search window up, and again right click and open each result in a new tab, do whatever I need to do, and then come back to the main search result list, and keep going. Some pointers for this phase:

• Some results will be ones you already discovered
• Other results will lead you to additional spam posts and/or users. Take care of each of them in turn (as above) and continue to add to your URL list.
• Scratch each spam URL off your list once you have worked your way through all the results for that particular spam URL.
• Some times you may have to "massage" the URL a bit if it is too close to common search terms
  • shorten to base URL
  • add quotes around it
  • prepend "https://www."
  • etc...

 

6. The End

 

Get yourself a cold, tasty, delicious adult beverage of your choosing. You earned it. You have made the world a slightly better place than it was this morning. You have Maintained Civilization. Etc. 

 

I think that just about covers it. Any questions, comments are welcomed.

 

One thing I would like to re-iterate, if you don't have the time / patience for this, just Flag the post(s) and leave it for someone else who does. I think it's better to handle these properly and thoroughly than to half-arse it, deleting few links and then handicapping our ability to "get them all" later on.

 

Thanks!

 

EDIT: As an example, I will include my working list, below. This is what I came up with in a couple of hours this afternoon, all starting from one link that @Igor sent me in IRC (this is a bulleted checklist and table from org-mode; names of non spammers vindicated in my search have been removed):

 

Spoiler

** Spam Links

   - [X] showbox.run
     - [X] 192168ll.onl
     - [X] 7zip.vip
     - [X] 9apps.ooo
     - [X] apkals.com
     - [X] apkals.com/veteran-mode-dual-apps-apk
     - [X] applock.ooo
     - [X] appsync.biz/dafont
     - [X] bluestacks.vip
     - [X] dataguru.in/company/search-list/list-of-companies-in-gurgaon-dlf-cyber-city
     - [X] kodi.software
     - [X] luckypatcher.pro
       - [X] downloadnox.onl
       - [X] vidmate.vet
       - [X] vlc.onl
       - [X] user: haileykhysen
     - [X] minimilitia.mobi
     - [X] mxplayer.pro
     - [X] notesandprojects.com
     - [X] notesandprojects.com/kiran-news-agency
     - [X] oovoo.onl/adam4adam
     - [X] pnrstatus.vip
     - [X] salezshark.com/sales-automation
     - [X] sarkariresult.onl
     - [X] shareit.onl
     - [X] showbox.bio
     - [X] showbox.kim
     - [X] tutuapp.uno
     - [X] ucbrowser.vip
     - [X] webmentorz.com
     - [X] webmentorz.com/bank/wren-and-martin-book-pdf

|--------------+----------|
| User         | Spammer? |
|--------------+----------|
| Bï Ĺý        | yes      |
| jhoney       | yes      |
| klimbo       | yes      |
| nour         | yes      |
| haileykhysen | yes      |
|--------------+----------|

 

 

Edited February 12, 2020 by TRS-80
add example, clarify, typo

[balbes150]

Need to limit the time for regular users to edit messages to 2-3 hours or 1 day.

 

I don't see any reason for regular users to be able to edit messages after 1-2 days.

[TRS-80]

10 hours ago, balbes150 said:

Need to limit the time for regular users to edit messages to 2-3 hours or 1 day.

 

I don't see any reason for regular users to be able to edit messages after 1-2 days.

 

I think that is a very good idea.

 

The forum software though has limitations what it can do. But site level Admin like @Igor and @lanefu have more access to different settings than we do, so we will have to ask them.

 

There are enough Mods around, I think, in the event some legitimate user might need to edit something in an OP or elsewhere for legitimate reasons, they could just Flag and do it that way.

Edited February 12, 2020 by TRS-80
clarify

[Tido]

16 hours ago, TRS-80 said:

However at a very minimum, you should be taking down a list of all these spam links you come across.

sounds like way to much work. Watch a Video on YouTube (undercover) how they work in those countries.

I know how I did it (as moderator), it was fast and quick.

 

10 hours ago, balbes150 said:

Need to limit the time for regular users to edit messages to 2-3 hours or 1 day.

 

I don't see any reason for regular users to be able to edit messages after 1-2 days

I can see the idea behind it. But once a spammer is flagged, it is game over.  Until the person gets a new profile.

 

KISS (is your friend)

[TRS-80]

3 minutes ago, Tido said:

sounds like way to much work

 

I don't care how much work it takes. I care about doing a proper job.

 

You are, of course, welcome to go off in a hurry with the rest of the world... It certainly seems the popular thing to do nowadays...

 

It actually doesn't take that long, once you get your method down and get into a rhythm.

 

Anyway, I am well aware of many people's time constraints nowadays, hence my re-iterating at the end:

 

16 hours ago, TRS-80 said:

One thing I would like to re-iterate, if you don't have the time / patience for this, just Flag the post(s) and leave it for someone else who does. I think it's better to handle these properly and thoroughly than to half-arse it, deleting few links and then handicapping our ability to "get them all" later on.

 

[Werner]

9 hours ago, Tido said:

I can see the idea behind it. But once a spammer is flagged, it is game over.  Until the person gets a new profile.

Sure it is. Though spammers have infinite resources. We do not.

 

20 hours ago, balbes150 said:

Need to limit the time for regular users to edit messages to 2-3 hours or 1 day.

 

I don't see any reason for regular users to be able to edit messages after 1-2 days.

+1

[Igor]

11 hours ago, TRS-80 said:

I think that is a very good idea.

 

Edit time restriction

 

Can edit own content up to X minutes after posting or “default" Unlimited.

1440 minutes?

[TRS-80]

3 hours ago, Igor said:

1440 minutes?

 

I think much less. Maybe a couple hours?

 

How often have any of you guys come back up to 24 hours later to edit a post? I rarely do much after the first few minutes (I always read after I post and often catch some typo or grammar).

 

The shorter the time, I think the easier for us to keep our eyes on new members we suspect to be spammers.

 

Remember, they can always Flag it of something really needs to be edited, or just add their changed thoughts to a new post.

[NicoD]

I also think that's a good idea. A few hours should do.
Only problem I see here is for people maintaining tutorials. They still need to be able to adjust things that have changed. 
No problem for me since I'm a mod. But there are others too. For sure @balbes150 can't be harmed by that rule for instance. 

If it can be only for people with low content count maybe? But that will make things more complicated of course.

[Igor]

32 minutes ago, NicoD said:

Only problem I see here is for people maintaining tutorials.

 

This settings is tied to a particular group. Now we have:

- validation (each posts must be approved)

- users (everyone)

and groups like developers, moderators, ...

We need something in between. We need to define (automatic) promotion rules and there are many ways to do that.

[balbes150]

If possible, it is good to have a group of " advanced users "(the name is conditional and you can use any). Adding to this group is done manually based on a presentation or discussion results. It includes users who can edit their messages without restriction (for example, those who write manuals or other payloads).

[Igor]

6 minutes ago, balbes150 said:

Adding to this group is done manually based on a presentation or discussion results.

 

This process could/should be automatised - by having more then one post (now it's just one approved and you are a member with all rights), answering some questionnaire, having n likes, minimal forum subscription, ... should help. We can't waste our resources to fight unlimited ones.

[Igor]

Additional measurements:

- link filtering -> Allow only the links specified: *.github.com, *.armbian.com, ... -> Action to take when a disallowed link is posted: "block" or "need approval mode"

- enable "Two Factor Authentication" by default

- Group Promotion Rule: If Content Count > 3 and Reputation > 1 or 2 (likes) move user to group -> "trusted member" with no edit limitations.

[guidol]

I often edit my posts... maybe for correcting a word, entry or a new command or link

and this sometimes days later.

 

I also got sometimes problems, that not all editing options are available when editing a post against the editing options while creating a new post.

 

In a other forum the where the timed editing option is enabled there is much "thread-overhead" because some people have to add something to the thread because they cant edit it anymore.....

They dont got spammers (I dont see there any) - maybe they check in a other way?

[guidol]

link filtering to special known domains only for new users?

because if I post instructions I also add a link to a page where I did find the info for my solution.

[Werner]

2 hours ago, Igor said:

 

This settings is tied to a particular group. Now we have:

- validation (each posts must be approved)

- users (everyone)

and groups like developers, moderators, ...

We need something in between. We need to define (automatic) promotion rules and there are many ways to do that.

 

1. Validation
   • automatically gained
     • immediately after registration and email verification
   • every posting needs approval
2. Newbie
   • automatically gained if
     • registered at least one day AND
     • has two approved postings
   • has permission to
     • post without approvial but
     • is limited to edit up to one hour after post creating
3. Member
   • automatically gained if
     • registered at least seven days
     • have five postings
   • has permission to
     • edit posts without limit

My suggestion.

 

 

From a technical point if view it is recommended to do such discussions unpublic since there are also advanced spamer that especially looking for such limits in postings or FAQs and such to adjust their bots matching each individual website.

 

 

[TRS-80]

I think very strongly that validating email should be (one of?) main steps to graduate from Validation -> Member. Anybody can put in whatever they want for an email address, it's all BS until they have validated it. And in the meantime they can be posting already...

 

Most of spammers I see, never even validated their email...

 

Some number of Likes (perhaps along with other criteria like post count, and/or time on forum) I think is probably a good idea for this "Trusted Member" (or whatever we want to call it) intermediate level. I think Likes especially are hard to fake.

[Tido]

On 2/12/2020 at 4:12 AM, TRS-80 said:

I developed to deal with these sort of spammers, once someone has Flagged

Before everybody continous

- what do you guys try to solve?

- is it worth to limit the users freedom  (@balbes150 , @TRS-80, @Igor limit the time for regular users to edit messages to 2-3 hours or 1 day) ?

or are there other ways ?

 

I personally get the impression the armbian forum is for outsiders a shark tank and you seem to try to make it worse, just because of a minority of spammers.

You know DRM, digital rights management. Where a few steal, but the majority of honest payers (people) get to struggle with copy-protection. Where do you see yourself?

Is it worth to sacrifice the freedom of edit?

 

[Igor]

1 minute ago, Tido said:

Is it worth to sacrifice the freedom of edit?

 

Freedom to edit will only be limited to users we don't know. Nobody will even notice.
 

[TRS-80]

3 minutes ago, Tido said:

Is it worth to sacrifice the freedom of edit?

 

Perhaps you post in haste, without reading through entirety of thread, which mostly discusses where and how to strike the correct balance.

[Tido]

13 hours ago, Igor said:

Nobody will even notice.

If you read the thread, look at @guidol 's   post.  Freedom is freedom.  Free as in beer, as they say for OpenSourceSoftware.  If you think different. Go ahead, full steam.

Edited February 15, 2020 by Tido
added a missing y

[Igor]

2 minutes ago, Tido said:

If you read the thread, look at

The plan is to put more limit to newbies, perhaps create another group between validating and users. No limiting normal users. This is how I see it, but other might see it different which is why we talk first and do action on some common grounds later. If most of us will think this is a stupid idea, we won't proceed.

[lanefu]

yeah basically just.. don't allow posts until email is validated?

[Igor]

Email validation is not good enough to fight spam. New users need to have one approved post .. but sometimes is hard to decide is it a bot, human or spammer. That's why we need something in between.

 

Typical spammer replies to few posts, then edit those posts and add links. We want to deal with that.

 

[TRS-80]

12 minutes ago, lanefu said:

yeah basically just.. don't allow posts until email is validated?

 

No. Anyone can still post any time, post(s) will just be moderated until email is validated. That was something I suggested. To include validating email as one of main criteria to go from Validating -> Member status. This is correct balance I think between (very) low friction to contribution, and other factors (moderation, spam, etc...).

 

Then there is separate discussion of possibly creating new group, slight step up from regular Member like "Trusted Member" or such who would have extended editing time (basically unlimited) for use case of maintaining tutorials, etc. like what @guidol outlined above. Which I support as I think that strikes the balance perfectly.

 

EDIT: Maybe better naming would be "Probationary" or "New" Member and then "Trusted" (or similar) to more clearly delineate the tiers? With the idea of course that anyone participating in the forum for any (relatively short) length of time, some posts, and (perhaps most importantly) 1 or more Likes, would be fairly easy to get to "Trusted Member" (or whatever we decide to call it).

Edited February 14, 2020 by TRS-80
add last bit

[Werner]

Quote

Email validation is not good enough to fight spam.

True. Email validation is probably the easiest thing spamer can bypass by either using trashmail services or script-based creating accounts from any freemail provider.

 

[chwe]

On 2/13/2020 at 3:12 PM, guidol said:

I often edit my posts... maybe for correcting a word, entry or a new command or link

and this sometimes days later.

well you should be moderator long time ago.. but that's a different story.. (btw. how did that not happen?)

 

for the rest of this post, take it with a grain of salt

 

7 minutes ago, Igor said:

but other might see it different

problem, most of those maintenance topics are boring as hell means that people may read the first post and then decide to no longer follow it, so once 'our' solution grows the majority of let's call them power users/ developers/ maintainers and even some moderators might not even realize which new rules are in charge now.. So the majority of the people contributing to the thread might be fine with the change.. Whereas it's not clear what the majority of power users etc. thinks about it...

 

On 2/13/2020 at 1:58 PM, Igor said:

 

This process could/should be automatised

an automatized solution to fight spam will halve a automatized counter to produce spam.. And I'm confident that they will have more resources than we have...

 

Cause I only saw a few of them (2-3).. About how many cases do we talk here? If we have only a few of such 'power spammers' I don't think it's worth to restrict user freedoms just to get those few spammers.. As soon as they edit recent topics of interest they get caught anyway and get flagged as spammers - game over for that account.. If we have a bunch of such spammers we could talk about restrictions but obviously they'll find a way to counter this so I don't see any automatized solution which will work in the long term..

 

26 minutes ago, lanefu said:

yeah basically just.. don't allow posts until email is validated?

back then when I maintained my own mailserver to register on various sites I had a small firefox plugin which generated a new mailadress random@mydomain.com and also 'validated' that mailadress successful for most cases without me taking any actions..  I'm quite sure their skill-level is higher than mine to achieve it.. I even made some statistics which sites then use those random mailadresses to send me spam after its..

 

As soon as a forum has some popularity you'll have spam, and you'll have to deal with. If the community is overall healthy they will hopefully help to report spammers or ignore those links (which mostly solves the issue itself, best case they report it then when they click by mistake on it - if the viagra for 2$ link isn't clicked - the spammer doesn't gain anything from it). If the community isn't healthy at all we better figure out what's wrong with our community than with the spammers..

[lanefu]

5 minutes ago, Werner said:

True. Email validation is probably the easiest thing spamer can bypass by either using trashmail services or script-based creating accounts from any freemail provider.

 

Maybe but... 4600 validating accounts disagree

 

[TRS-80]

4 minutes ago, Werner said:

Quote

Email validation is not good enough to fight spam.

True. Email validation is probably the easiest thing spamer can bypass by either using trashmail services or script-based creating accounts from any freemail provider.

 

OK, even if that is true, I don't understand why not to do it? It requires zero effort on our part? And makes one more hurdle for spammer, but not legitimate user.

 

1 minute ago, lanefu said:

Maybe but... 4600 validating accounts disagree

 

Thank you!. This is what I have seen as well, hence my arguing so strongly for it.

 

There is "theories" of what spammer could do, and then there is data (that we have) of what they are actually doing...

[chwe]

21 minutes ago, TRS-80 said:

There is "theories" of what spammer could do, and then there is data (that we have) of what they are actually doing...

can you provide data how many spammers overcome our current system with first thread approved editing their follow up and how may posts they modify after its?