Advice on performance for external API query
Hi all,
I'm looking for a bit of advice on improving the performance of a very simple API call that I've used to develop a Joomla CMS Module in PHP for use on a client's main website.
The purpose of the module is to display the 3 most recent posts for a specified "Custom Tag" which the Community admins apply to various posts.
We have two categories in our community, and the module displays posts from both categories so there are 2 API calls as follows:
http://saneforums.org/restapi/vc/categories/id/lived-experience-forum/posts/for/metadata/key/modbar.anxiety/value/true?message_viewer.topic_sort_order=last_post_date&restapi.format_detail=full_list_element&restapi.response_style=view&page_size=3
and
http://saneforums.org/restapi/vc/categories/id/carers-forum/posts/for/metadata/key/modbar.anxiety/value/true?message_viewer.topic_sort_order=last_post_date&restapi.format_detail=full_list_element&restapi.response_style=view&page_size=3
Each API call is loaded in to a separate PHP variable which I then use PHP's Simple XML Parser on to extract the data I need to display in the module.
You can look at the module in action here (down on the right hand side, heading "From SANE Forums"):
https://www.sane.org/mental-health-and-illness/facts-and-guides/anxiety-disorder
The problem is that the pages which have the module on them are loading very slowly at around 8 to 10 seconds while all of other pages load instantly. The site is hosted on a pretty serious setup at Rackspace so it's not a general hosting issue, and as I said it only happens on pages with the module assigned.
So I can only assume that the page load delay is coming from the query to the Lithium API and I'm wondering if anyone has any advice on that front.
One option I've thought of is to have a separate script activated by Cron which calls the API and generates a static XML file which the module would access instead of doing the real-time query - does that sound like a possible solution?
Would appreciate any advice that anyone might have.
Thanks
Nathan
snaffle a few elaborations on the points made above:
a) Cache the output of your Lithium API call, to do that you have to create an endpoint where you could also merge the data from the two calls into one dataset and then cache it for later reuse, which you do with something like that
<#assign data = appcache.set("externalfeed", yourdatainavariable) />Depending on how frequent the data should refresh, you have to contact support so they can adjust (lower) the cache expiry time for the appcache. In your endpoint you can then check if there is something stored for that cache-key and if so return that, else make the API calls and regenerate the data, e.g.
<#if appcache.get("externalfeed", "")?has_content> <#-- output your data, be aware that you cannot just print freemarker node objects, so you have to convert them somehow to json/xml, which is not easy btw... --> <#else> <#-- re-fetch the data and put it into the cache --> ... <#assign data = appcache.set("externalfeed", yourdatainavariable) /> </#if>But this is basically putting the work from your PHP script to Lithium which requires some FreeMarker/Lithium API knowledge.
If we say that is not available then options to maybe slightly improve the performance is
a) to simply use AJAX/JavaScript to fetch the content from the PHP script after the page has initially loaded and inject it to the right place, so the main content will be there while the AJAX call is getting the "slow" content from Lithium. This would be the easiest solution but doesn't actually change the performance, it just changes the perception =).
b) Go away from parsing the response XML in PHP and just use the restapi.response_format=json URL parameter for your API calls that will return JSON instead of XML, which you can parse much more efficiently in PHP with json_decode(data, true); but I guess this will not significantly improve the performance of the whole thing...so just a micro-optimization and more elegant solution.
if you have FreeMarker knowledge you can try to use API v2 message resource together with the restd context object inside an endpoint, the restd context object can actually return the data as a JSON string inside FreeMarker code (but you will not be able to modify the fetched data when doing that!) and API v2 would maybe allow to combine your two calls into one with an appropriate query, which means you can just cache and output the response without further worries about how to output it...
