Pagination in Search API v2

Hello prasath,

It looks like you’re facing a common issue with cursor-based pagination. When using the next_cursor token, it’s important to ensure that each subsequent request is fetching new data rather than repeating the initial results.

import requests

# Initialize variables
base_url = "https://[COMMUNITY DOMAIN]/api/2.0/search"
query = "SELECT distinct id FROM users where last_visit_time >= '2024-02-01T00:00:00' order by id asc"
headers = {'Authorization': '<access_token>'}
params = {'q': query}
next_cursor = None

# Function to fetch data with pagination
def fetch_data(cursor):
    if cursor:
        params['cursor'] = cursor
    response = requests.get(base_url, headers=headers, params=params)
    if response.status_code == 200:
        data = response.json()
        next_cursor = data.get('next_cursor')
        users = data.get('users')
        # Process your data here
        return next_cursor, users
    else:
        print(f"Error: {response.status_code}")
        return None, []

# Loop to handle pagination
while True:
    next_cursor, users = fetch_data(next_cursor)
    if not users or not next_cursor:
        break
    # Add your logic to handle the users data

This script uses a function to handle the fetching of data and includes the cursor in the parameters only if it exists. The loop continues to call this function with the new next_cursor until there are no more records to fetch.

Remember to replace [COMMUNITY DOMAIN] with the actual domain and <access_token> with your valid authorization token. Also, ensure that the next_cursor is being correctly extracted from the API response and that the API endpoint supports cursor-based pagination as expected.

If you continue to face issues, it might be helpful to check the API documentation for any specific requirements or limitations regarding pagination.