Testing Checklist and Troubleshooting
Testing checklist
|
Check |
How to test |
Expected result |
|
Prerequisites met |
Verify AI Settings + models selected, Spam Management enabled |
AI Moderation toggle available |
|
Confidence threshold |
Post borderline content |
Above-threshold auto-actioned; below goes to Moderation queue |
|
Moderation style |
Post in a Required Review board |
Post hidden until approved |
|
AI verdict displays |
Check Moderation tab |
Confidence scores shown in AI Verdict column |
|
Analysis modal |
Click ⓘ next to any verdict |
Verdict, reason codes, and reasoning shown |
|
Reason codes |
Check rejected content as a standard user |
Correct reason code message displayed |
|
Custom Reason Code |
Create a code; post matching content |
AI uses the custom code to categorise content |
|
Board context |
Post jargon with vs. without board context |
Board with context more lenient on jargon |
|
Activity limits |
Post above configured limit in a short window |
User rate-limited; subsequent posts blocked |
|
Appeal submission |
Reject a test post; appeal as an end user |
Appeal appears in Manage Content Appeals tab |
|
Appeal approval |
Approve an appeal in the queue |
Content visible; user notified |
|
Max appeals |
Submit maximum appeals for one post |
Appeal button disappears after limit reached |
|
Abuse notifications |
Submit an abuse report |
Configured email receives notification |
|
Content Filters |
Post a "Do not allow" term |
Post blocked regardless of AI outcome |
Troubleshooting
|
Symptom |
Most common cause |
Fix |
|
AI Moderation toggle disabled |
AI Settings or Spam Management is not enabled |
Enable both; select AI models |
|
Content not being analyzed |
Board on Auto-Approved, or AI Moderation off |
Set style to Selective or Required; confirm AI Moderation is ON |
|
Too many false positives |
Threshold too low or board context missing |
Raise threshold toward 75%; add board context with examples of acceptable content |
|
AI not flagging violations |
Threshold too high |
Lower threshold toward 75%; verify content actually violates policies |
|
Appeals not in queue |
Appeals not enabled or filter set incorrectly |
Enable Appeals System; set filter to "Pending" |
|
Users can't see Appeal button |
Max appeals reached or missing permission |
Check appeal history; verify user role has appeal permissions |
|
Board settings not applying |
Still inheriting from Global |
Toggle "Use {board} specific {setting}" ON and save |
|
AI missing obvious violations |
Community Guidelines too vague |
Refine guidelines with specific violation examples |
|
Abuse reports not arriving |
No email in Abuse Notifications |
Add a valid email for the relevant content type |
|
Content Filter not blocking |
Term missing or action is "Take no action" |
Add term; set action to "Do not allow" |
|
Queue overwhelming |
Threshold too low or too many Required Review boards |
Raise threshold; move lower-risk boards to Selective Review; add role-based overrides |
Adjusting the confidence threshold:
Raise it (toward above 80%) if you see too many false positives, over-moderation complaints, or are overturning more than 30–40% of AI decisions.
Lower it (toward below 70%) if violations are slipping through or moderators are finding obvious issues in approved content. Wait at least 1–2 weeks between adjustments to gather meaningful data.