Forum Discussion

keithkelly's avatar
5 months ago

2) Which Way is FRT Calculated? (topic or reply?). **EDIT: Appears to be Topic?? Details added:

In Community Analytics, is First Response Time calculated like:

  1. Select all TOPICS started in January, and calculate their FRT, ..or..
  2. Select all FIRST REPLIES posted in January, and calculate their FRT

---

**UPDATE:**

History:

  • A year ago, we noticed avg_thread_response_time seems to be returning "total thread response time."
  • So, my brilliant coworker Kenton wrote a hardcore freemarker function to calculate the average.  And it OFTEN aligns w/ Community Analytics:

 

I mean, that's dang good.  But leaves a few questions that led to a bit of a WTF moment.  (friendly WTF moment.. just trying to wrap my head around the FRT stuff):  Why are some boards really far off?  (so, I investigated...)

...

Looking into our internal Policy forum - 

SystemAverage Response Time (minutes)
Community Analytics1,366
Kenton's Whizbang FRT Function5,426
Community Analytics Export:  (SS below)5,426 🧐

 

Here's the CSV export from Community Analytics:

 

The yellow stuff verifies Kenton's "Only get July's Topics" approach, which starts by getting all TOPICS in that date range:

 

 

<#local topics_query = "select replies, post_time from messages where board.id = '${board_id}' AND depth = 0 and replies.count(*) > 0 and post_time > ${start} and post_time < ${end} limit 5000" />

 

 

And don't forget, this aligns with almost every other CA stat...😅

 

Let's talk about the Blue Box (old topics that received replies during this time period):

  1. I propose those in the blue box (where replies == 1) should be factored into the Average FRT...
  2. BUT, that would INCREASE the Average FRT to 26,617.  See?

 

Let's update the table:

SystemAverage Response Time
(minutes)
Community Analytics1,366
Kenton's Whizbang FRT Function5,426
Community Analytics Export:  (July topics only)5,426 
Community Analytics Export: 
(Including blue where replies == 1)
26,617 🤦

 

Where is Community Analytics getting 1,366 from?  (I feel like I'm missing something obvious).  Unsettled, I tried a theory that got me CLOSE but not exact like previous calculations...

👉THEORY:  Maybe CA is excluding the extremes?  I tried a lot of exclusion scenarios, and the closest I got was this:

  • Exclude everything above 10,000 mins, and
  • Exclude everything below 20 mins

 

This got me pretty close -

.  

1,344 minutes is close to 1,366 minutes, but is very unconvincing.  

 

Back to the table:

SystemAverage Response Time
(minutes)
Community Analytics1,366
Kenton's Whizbang FRT Function5,426
Community Analytics Export: 
(July topics only)
5,426 
Community Analytics Export: 
(Including blue where replies == 1)
26,617 
Community Analytics Export: 
(but ignoring under 20mins and above 10k)
1,344 😂

 

 

So what's right?

  • I vote 26,617 is accurate It's the only one that truly considers all first replies.  
  • Doesn't the "Topics" approach create a situation where many first replies are never considered?  Those in the blue box weren't considered in their respective months b/c they didn't have replies.  And they are certainly not being considered now.    -- even if I retrospectively run the report from April 2023, I get an Average FRT of 743 minutes, which is clearly not considering that 678,620 minute response time ...
  • But, I think the 5,426 is more relevant for day-to-day reporting and decision making.  While it's technically inaccurate (e.g, because it leaves out topics that are asked in March but answered in April),  it avoids the extreme outliers that slip through the cracks and skew the picture.  

What do you think?  Is there like some simple button I'm forgetting to push? 🤪

 

No RepliesBe the first to reply