Forum Discussion

keithkelly's avatar
3 years ago

Can I "Trace" the Requests Happening Behind the Scenes?

For the products we build, we have trace tools that help us understand what APIs etc are being called behind the scenes.  We even give them to our customers (b2b) so they can analyze & learn.

Of course there's Fiddler, which does quite a bit of tracing, but I *think*(?) Fiddler is irrelevant for tracing Khoros API stuff - but it's been probably 5 years since I've used Fiddler seriously. 

Are there any Trace tools available (either by Khoros or that expose Khoros goodies) that can help make sense of the API requests called by various components? 

(example:  "My Group Hubs" component ... what API call is happening behind the scenes there?  I'd like to build my own component that grabs the same list but uses it differently.  I've spent 1hr in the docs, and haven't found what I'm after.   Buuuutttttttttttt if I could only trace the dang thing........)

Does anyone have any tracing insight?  Are there any tools available? 

14 Replies

  • luk's avatar
    luk
    Boss
    2 years ago

    Well, I'm guessing as much as you on what those older components use as their datasource and yes, I've struggled in the past to re-build "better" versions of out of the box components due to the API not covering all the data/functionality we need to actually do it. BUT, of course there is a "but", there's a pretty ugly way of doing it anyways. The gist of it is, you make an @override of the OOB widget, assign <@delegate /> to a variable, so you get the original HTML markup of the OOB widget in a variable, then you regex the s*** out of it and extract information you can then use to make API calls, usually the ID does it, e.g. in your scenario you'd regex out the user id's for the kudos leaderboard, then you create API requests for using those rendering your own leaderboard... It is of course ugly because first it loads the OOB widget (using server time for that I assume) and then you come in, regex stuff out, then make additional API calls to build your own, so I'd definitely recommend using <@liaMarkupCache /> in those kind of "solutions".

    But before you do that, have a look at API v1, for example here https://devdocportal.khoros.com/t5/Community-API-v1-Reference/bd-p/restv1docs?section=commv1&leaf-id=Blog.kudos.authors.leaderboard#Blog.kudos.authors.leaderboard

    Hint: just type "leader" into the searchbox to find API endpoints for various leaderboards. My guess is that a lot of older OOB widgets still use API v1, as that was the only thing that existed back in those days.

    EDIT: Went digging in some old codebases because i wondered what it was I did this for, here you go (had to adapt the code slightly as it used functions etc. defined elsewhere, so it's untested, but that's the bulk of it anyways):

    	<#--
    	/**
    	/*	There is no leaderboard API available for top solution authors (there is for kudos...)
    	/*	and it's nearly impossible (only by querying ALL users and then their authored solutions
    	/*	with "SELECT * FROM messages WHERE is_solution = true AND author.id = <id>") to do by ourselves.
    	/*	So we parse the studio component widget for top solution authors and crank out the user ids listed
    	/*	there, which we then can use to actually fetch the user objects.
    	/*/
    	-->
    	<#-- ids will finally store the extracted user ids -->
    	<#local ids = utils.sequence.builder() />
    
    	<#assign cmp>
    		<#assign markup>
    			<#-- @delegate seems to always be available, but can be empty! -->
    			<@delegate />
    		</#assign>
    
    		<#if !markup?has_content>
    			<#attempt>
    				<#assign markup>
    					<@component id="solutions.widget.accepted-solutions-leaderboard-taplet" />
    				</#assign>
    			<#recover>
    				${"including markup from @component 'solutions.widget.accepted-solutions-leaderboard-taplet' failed!"}
    			</#attempt>
    		</#if>
    
    		${markup}
    	</#assign>
    
    	<#-- Extract required data from original component markup, regex101.com can be your friend! -->
    	<#assign matches = cmp?matches("\\/user-id\\/(\\d+)") />
    
    	<#list matches as match>
    		<#-- Only add a match once -->
    		<#-- The ?groups build-in contains a sequence with [0] = the whole match and [0+groupn] items with their match -->
    		<#if !ids?seq_contains(match?groups[1])>
    			<#assign ids = ids.add(match?groups[1]) />
    		</#if>
    	</#list>
    
    	<#-- then get user objects from all extracted ids -->
    	<#assign query = "SELECT * FROM users WHERE id IN('" + ids.build()?join("', '") + "') />
    	<#assign response = rest('2.0', '/search?q=${query}') />
    	<#-- do someting with the data -->
    	<#list response.data.items as obj>
    		
    	</#list>
  • cblown's avatar
    cblown
    Boss
    2 years ago

    I did catch a glimpse of some source code a long time ago at one of Lithium's conferences while talking to one of the lead devs at the time, complaining about this very issue. And yes it certainly wasn't FreeMarker ðŸ˜‰ 

  • mdfw's avatar
    mdfw
    Genius
    2 years ago

    "regex the s*** out of it" is the best way to code. 🙂

    Seriously, thanks for the hints. Doesn't really answer the original poster's question, but it does help me.

  • Glad it helped someone =)!

    And yes, it doesn't answer the OP's question, that was done before: Basically, no, it's not possible to trace API requests from OOB components as some of them seem to use some kind of internal magic to access data we cannot via API v1 or v2.

    And for your usecase, the kudos leaderboard, there is actually an API v1 endpoint for that!