Spam false positives

[deads]

Are there specific details on how the false positives are handled? In Akismet, false positives can be correct by "Submitting Ham" back to the API so that future look-alikes get a lower potential spam rating.

From what I can find about the spam tool in Community:

Show more

Spam Management

This feature is available out-the-box at no charge and is the most powerful tool in the arsenal. When enabled, spam management will automatically monitor the community and automatically flag content as spam, then send it to the spam quarantine. The feature extracts data from both a local (your community) and global (all Khoros communities) spam repository where the information is used to improve the algorithm used for spam detection. It won't be perfect immediately and will require close monitoring + manual correction, but this is a learning tool that adapts with each correction and will eventually stabilize with a high detection rate once it has enough data. You can make users immune from all spam grading by granting them the "Bypass automatic spam rejection" permission.

It's not totally clear to me what this means. I want to believe some voodoo is occurring behind the scenes, but we see a fairly consistent false positive rate around 30% that doesn't seem to be going down over time. 

[CarolineS]

Hi @deads - 

This isn’t an answer to your question, but just chiming in to say that we had so many false positives with the spam filter (and very few actual spam posts) that we turned it off. The false-positive problem was exacerbated by not having alerts when things are sent to the spam quarantine.

Hope this is a useful data point for you.

Cheers!

[tyw]

Just chiming in to echo the need for Spam Quarantine notifications so false positives can be restored proactively instead of reactively. 

We can't turn off the feature due to the volume of spam we receive so this false positive issue ends up costing us money when users contact us to inquire why their post was marked as spam. 

I know this doesn't really help but perhaps Khoros can provide more insight - does their system help lower the FP rate, or remaining at 30% due to spammers increasing, as well as changing their tactics? 

[deads]

We've had similar situations of customers reaching out through other support channels because they can't post on Community due to false positives. Not just escalations on the Community itself!

I've re-calculated our false positive rate under the assumption that all manually-marked spam is not a false positive. That brings our false positive rate (based on auto-classified spam alone) up to 40%.  And under this model, the false positive rate has actually increased over time.

[JasonHill]

One of the problems we're seeing with false positives is when users choose to edit their post a couple of times a legitimate post then gets marked as spam. So it doesn't seem to have to do with the content of the post, but rather the action of editing it multiple times.