Forum Discussion

synthesis's avatar
synthesis
Guide
12 years ago

How do I get older/more records with the REST API?

I work for HP. We have a gazillion product forums. I wanted to harvest a bunch of data with a call like this:

 

http://h30499.www3.hp.com/restapi/vc/categories/id/bsc-412/posts/recent?page_size=1000&page=1&restapi.response_format=json&restapi.format_detail=full_list_element&restapi.response_style=view

 

The max I can get at a time is 1000 records. If I keep increasing the "page" parameter it starts returning me zero results when page equals 3.

 

What if I want to pull 10,000 records?

 

If 2000 is the max, how do I get to older posts because 2000 records only takes me back to June of this year for the products I'm interested in.

 

Is there another way to bulk request more records?

 

I understand why you'd want to have a cap, but how do I work around it?

 

Thank you!

  • dementad's avatar
    dementad
    Genius
    12 years ago

    If you put your logic in a loop like this, you can make y as large as you want. It will automatically end the loop when you run out of data.

     

    <#list 1..3 as y>
    <#assign username = restadmin("/users?page_size=1000&page=${y}") />

    ....

    </#list>

     

    This may not be the optimal way to do this. I'm very new at this, but it works.

    • synthesis's avatar
      synthesis
      Guide
      12 years ago

      >> If you put your logic in a loop...

       

      That is is exactly what I'm doing. It doesn't work for me.

       

      We have two Lithium boards I'm working with: One for commerical products and one for consumer. They BOTH return 0 results on any call with "page_size" greather than two (3 or greater). I KNOW there are more records than 2000 because we've been using Lithium for a long time and have products that go back many years.

       

      This still smells of an API cap.

       

      What else can I try/do?

      • dementad's avatar
        dementad
        Genius
        11 years ago

        I just ran the loop on a subset of my users in an endpoint and got 2794 records back. I'm not running into this limit.