AI Comment System:Boosting Engagement by 25%
A moderation queue that used to fight comment managers now works with them, and the business felt it too.
I designed Commod, an AI-assisted moderation system for Mako's comment section. An AI first pass triages tone, spam, and duplicates, routing high-confidence cases automatically and grouping uncertain ones by reason so moderators only review what needs judgment. I was the only designer on the project. I led the research, including 5 in-depth interviews with comment managers, led the design through interaction flows and shipped screens, ran 5 usability sessions, and built the product's design system.
Finding the Real Bottleneck
Moderation Couldn't Keep Up
Online discussion keeps Mako readers engaged, but moderating it at scale was unforgiving: thousands of comments could hit the queue within minutes during major news events. Comment managers had no real tooling, just batch-refreshed pages, confusingly labeled section channels, and manual, one-by-one spam and duplicate catching. Hiring more people was the only lever.
Talking to the People in the Queue
I ran this through the Double Diamond: Discover, Define, Develop, Deliver. Instead of assuming what slowed comment managers down, I started with five in-depth interviews across different news desks. Affinity mapping of their actual workflow surfaced friction points no dashboard metric would have shown me.
“It takes me more time to understand if the comment is reasonable for the article.”
The Role the System Was Built Around
Reads every incoming comment and approves or rejects it, at speed, across breaking news and routine articles.
- 01Continuous scrolling without batch limits or lost progress
- 02Clear article and channel context, so decisions stay consistent
- 03Obvious spam and repeat offenders filtered out before manual review
Affinity mapping from the five interviews, grouped into habits, pain points, direct quotes, feelings, and past experience.
Habits
Pain points
Strong quotes
Inner feelings
Past experience
4 Pain Points, Straight From the Queue
Duplicate & Repeat Spam
No tools existed to filter duplicate comments from the same IP, so the same spam got manually rejected again and again.
System Limitations
Batch comment loading and unclear channel labels made the basic mechanics of moderating harder than they needed to be.
No Visibility Into Performance
Managers had no data about the site or their own performance, no way to see the bigger picture between shifts.
Distrust in Automation
Past experience with blunt, keyword-based AI filters left managers skeptical that automation could actually help.
Mapping a full moderation shift, from login to sign-off, surfaced exactly where the friction was.
- 1
Prepare for review
ConfidentFocusedTasks- Log into the dashboard
- Organize shifts with a colleague
- Review moderation rules
OpportunityBetter onboarding and performance tracking
- 2
Navigate to the right section
FrustratedDisorientedTasks- Search for the article
- Verify the channel name
- Flag priority comments
OpportunityClearer labeling of article names and channels
- 3
Review the first batch
OverwhelmedDeterminedTasks- Read through each comment
- Approve or reject comments
- Refresh after every 50 comments
OpportunityContinuous scrolling to avoid frequent refreshing
- 4
Handle inappropriate comments
EmpoweredDrainedTasks- Flag inappropriate comments
- Check repeat offenders by IP
- Escalate when necessary
OpportunityAdvanced filters for spam and repeated offenders
- 5
Complete the process
AccomplishedRelievedTasks- Submit the article for publishing
- Log performance metrics
- Exit the system
OpportunityAutomated performance report generation
No Direct Competitor, So I Studied the Category
Commod is internal newsroom software, so there was no true competitor to benchmark against. I worked hands-on inside a competitor's console and mapped the category through desk research, then decided which patterns were worth borrowing and which ones to reject.
- Third-party engagement platforms
- Batch pages, roughly 50 at a time
- Newsroom moderation tools
- Paginated queues
- Third-party engagement platforms
- Click out to the article
- Newsroom moderation tools
- Thread view
- Third-party engagement platforms
- Manual word and IP blocklists
- Newsroom moderation tools
- ML spam scoring
- Third-party engagement platforms
- Auto-hide at a threshold, opaque to moderators
- Newsroom moderation tools
- Auto-moderate at a confidence threshold
- Third-party engagement platforms
- Generic language support
- Newsroom moderation tools
- English-first
- Third-party engagement platforms
- Any publisher, any scale
- Newsroom moderation tools
- Community manager
Landscape synthesized from category research (G2 commenting systems, WAN-IFRA “Online Commenting”) and hands-on access to a competitor's console. Named platforms are illustrative of each category, not individually audited.
For the console itself, I looked at B2B SaaS products rather than comment tools: Remote, Sprig, Clerk, and Cake Equity. Their dashboards had already solved table density, filtering, and bulk actions for people who work in them all day, which is exactly what a moderator on shift does. My note on the research board said it plainly: don't reinvent the wheel.
I branched the flow at “View selection” so moderators can pull comments themselves or let the system push them, then funneled both paths into one repeating review loop.
More People Wasn't the Fix
The obvious answer was more moderators. The research said otherwise: the bottleneck wasn't judgment, it was that every comment demanded the same manual attention, however obvious. So I split the work. An AI first pass handles spam, duplicates, and clearly safe comments, protecting moderator attention for what actually needs a human. The speed came from redesigning the workflow, not adding headcount.
One System, Every Screen
Every screen in Commod pulls from the same foundation, and I built it as the product took shape rather than documenting it after the fact. That discipline is what makes approved green mean approved green everywhere a moderator looks.
Each state gets its own color, so the queue reads at a glance.
Blue 400 is the anchor: every button, active nav state, and link traces back to it.
Nine steps of gray carry every background, border, and line of body text.
All values are Commod's real product tokens, pulled directly from Figma. Single theme: no dark-mode variant exists for these. 47 color tokens across 5 ramps.
The System, Screen by Screen
The Home Page: a central hub for the user to access all the insights needed
- 1
Performance, at a glance
The Activities chart shows approved vs. rejected trend by day.
- 2
AI Assistant transparency
967 of 1,534 reviewed, split into approved, rejected, and needs-review, right on the home screen.
Moderating pending, approved, and rejected comments with full article context
- 1
AI insight tags
Respectful / Top-Ranking vs. Restricted Words / Spam surface before you read the full comment.
- 2
One-click source
A direct link back to the originating article.
Viewing comments with focus on one article
- 1
Full article context, no tab switching
The article and headline sit right next to the queue.
- 2
Confidence at a glance
A simple % ring shows approval likelihood before the manager even reads the comment.
Managing user activity and comment history
- 1
Activity tiering
Super Active / Casual / New / Active tags turn a flat list into a prioritized one.
- 1
History before verdict
A user's full approved and rejected record sits beside their details, without leaving the queue.
AI-assisted moderation with smart insights
- 1
Grouped by uncertainty type
Split into Maybe Spam, Potential Controversial, and Unclear Language, one call type at a time.
- 2
Suspicious text highlighted inline
The yellow highlight marker flags the exact phrase that triggered AI concern.
The AI Sorts, Moderators Decide
The AI protects moderator judgment instead of replacing it. It runs a first pass on tone, spam, and duplicates, routes the high-confidence cases through, and groups the uncertain ones by why they're uncertain.
Validating With the Real Users
I ran 5 usability sessions with the same moderators from research, closing the loop to validate whether the design solved their real workflow problems.
Strong First Impression
All 5 moderators reacted positively on first use, with one calling it “a game changer”.
Context Without Leaving
The inline thumbnail and article link erased the #1 pain point, no separate tab needed.
Faster Borderline Calls
Borderline decisions sped up noticeably once moderators could see the AI scoring in context.
They Want Benchmarking
Moderators asked to compare their rates against teammates, a request now on the post-launch roadmap.
What Changed for the Business
Redesigning the workflow around AI-assisted triage paid off across the metrics that matter to the business, not just to moderators. Comments publish faster, moderators handle a heavier load at a more consistent standard without adding headcount, and that consistency and speed gave readers enough reason to engage more.
If I Have More Time...
With more runway, I'd bring comment managers into testing earlier, since trust in the AI was the hardest thing to earn and more hands-on time before launch could have closed that gap faster. It confirmed my biggest insight: AI UX is really about transparency, people trust a decision because they can see why it was made.
Want to hear the longer version?
I'm happy to walk you through decisions, trade-offs,
and what I'd do differently.