Issue: #3
Type: BUG
Area: REST API v2 | FreeMarker
Collection: all
Platform Version: <= 19.7
Summary: FreeMarker context object apiv2.toJson() fails with "augmented" API response objects.
Ping: DougS
ISSUE SUMMARY: If an API v2 response object is modified (after fetching it) with FreeMarker, the apiv2.toJson() method fails with an error.
Due to a variety of shortcomings with API v2 we still have to rely on API v1 calls either because the data is completely unavailable with v2 or due to performance reasons (e.g. v1 is much faster than aggregating the same data with v2).
As we try to work primarily with API v2, we often fetch the necessary data with API v1 and then modify an existing API v2 response object with the data from v1 (this is the "augmenting" part mentioned in the summary, can also be thought of as "extending" the object or overwriting existing properties/values).
Doing this helps keeping the code more consistent/reusable (because field names in v1 and v2 differ in many cases which would complicate things when reading properties/values while rendering the markup).
The problem I came along when doing this kind of response object "augmentation" relates to the FreeMarker context object apiv2 which has a toJson() method that allows printing the response object as JSON which is very useful for debugging, e.g. seeing what is in the actual response within the component which can be different from what we get in the API Browser due to permission differences when logged in as an Administrator.
Trying to use apiv2.toJson() on a modified response object will throw the following error:
No compatible overloaded variation was found; declared parameter types and argument value types mismatch.
The FTL type of the argument values were: extended_hash (wrapper: f.c.AddConcatExpression$ConcatenatedHashEx).
The Java type of the argument values were: freemarker.ext.beans.HashAdapter.
The matching overload was searched among these members:
- lithium.template.FreeMarkerRestMethods$RestV2ResponseParserMethod.toJson(List),
- lithium.template.FreeMarkerRestMethods$RestV2ResponseParserMethod.toJson(lithium.api.lion.RestCallTemplateModel)
Tip: You seem to use BeansWrapper with incompatibleImprovements set below 2.3.21. If you think this error is unfounded, enabling 2.3.21 fixes may helps. See version history for more.
A practical example/use case of this technique is the following:
Goal: We need the total count of kudos for a topic, that includes kudos given to replies!
API v2 unfortunately is inconsistent here with regards to v1 as there is NO direct way (at least afaik) to get the sum of all kudos for a topic aggregated from topic message and replies to it. Although indirectly this is still doable with v2 by:
1. Fetching the kudos from all messages related to a topic with:
SELECT kudos.sum(weight) FROM messages WHERE topic.id = '4701' AND kudos.sum(weight) > 0
2. looping trough all the kudos objects and summing up their kudos values with:
<#assign kudos = 0 />
<#list response.data.items as obj>
<#assign kudos = kudos + obj.kudos.sum.weight?number />
</#list>
BUT the v2 call ALONE (without the looping) is roughly 15x slower than fetching the same value with API v1:
<#assign kudos = rest('/threads/id/4701/kudos/count').value?number />
Considering we most likely do that for each topic in a collection, a performance discrepancy like this quickly adds up and seems unnecessary. An ideal-world solution would be if we could simply specify topic.kudos.sum(weight) (which we can today, it just does NOT return the total count but only the kudos count of the topic message...) and get the aggregated value in one API call with all the other needed data. => FEATURE REQUEST
I hope that explains the performance part and why we are modifying v2 response objects, which works fine on a data level (e.g. we have no issues accessing the "augmented" properties/values when looping over the response object for rendering markup.
A simple example to reproduce the issue at hand:
<#-- fetch total kudos count for topic with v1 -->
<#assign query = "/threads/id/4701/kudos/count" />
<#assign count = rest(query).value?number />
<#-- fetch topic message object with v2 -->
<#assign query = "SELECT id, kudos.sum(weight) FROM messages WHERE id = '4701'" />
<#assign response = liql(query) />
<#-- augment the response object with total kudos count from v1 call -->
<#assign response = response + {
'data': {
'items': [
response.data.items[0] + { 'kudos': {'sum': {'weight': count}} }
]
}
} />
<#-- accessing the overwritten value works -->
${response.data.items[0].kudos.sum.weight?number}
<#-- but this throws an error -->
<#attempt>
${apiv2.toJson(response)}
<#recover>
${.error}
</#attempt>
