I’m trying to get pagination to work, but I keep getting the same results when I adjust the “after” filter in my request

Questions & Help
1

Hi thanks for your patience with all of my API questions. I’m trying to get pagination to work, but I keep getting the same results when I adjust the “after” filter in my request. I also noticed that the “after” filter takes in a string, not an int. I get this error when I try to run it as an int:
Output:

Out[29]: {'error': "Query does not pass validation. Violations:\n\nExpected type 'String', found '1000'. String value expected (line 9, column 42):\n              results(first: 500, after: 1000) {\n

This is my request when I try to go forward in my search results. I’m getting the same locations every time.

import requests
import json

def sf_gql_request(params):
    query = """
        query($postal_code: String, $start_date: DateTime!, $end_date: DateTime!) {
          search(filter: { 
            address: {
              postal_code: $postal_code
            }
          }){
            weekly_patterns(start_date: $start_date end_date: $end_date) {
              results(first: 500, after: "1000") {
                pageInfo { hasNextPage endCursor }
                edges {
                  node {
                    placekey
                    location_name
                    postal_code
                    raw_visit_counts
                    normalized_visits_by_total_visits
                    normalized_visits_by_state_scaling
                    date_range_start
                    date_range_end
                  }
                }
              }
            }
          }
        }
    """

    req = requests.post(
        'https://api.safegraph.com/v2/graphql',
        json={
            'query': query,
            'variables': params
        },
        headers=headers
    )

    return req.json()
params = {
    'start_date': '2020-01-06',
    'end_date': '2020-01-11',
    'postal_code': '94588',
    'after': '500'
}
ptown = sf_gql_request(params)
ptown
2

We are having a similar issue too, just adding myself to the reply thread in case there’s a solution

3

So to get next page you need to submit the endCursor of the previous page in the after . So for the first request omit after or submit empty string and then the next page can be accessed the value in the response payload. Looking at the script, the endCursor should be found at ptown['data']['search']['weekly_patterns']['results']['pageInfo']['endCursor'] . Use that value in after to get the next page

4

Do you know if the api supports the “offset” method described here? https://graphql.org/learn/pagination/

I looked in the safegraph api docs and couldn’t find anything

5

Unfortunately not at this time.

6

Got it, no prob, thanks. I’ll use the method you mentioned above

so the endCursor doesn’t appear in the json response we’re getting. Any thoughts on why not? Here’s the code

query  <- paste0(‘query {
  search(filter: {
      address: {
          postal_code: '20002'
      }
  }){
      places {
          results(first: 1) {
              edges {
                  node {
                      safegraph_core {
							placekey
              latitude
              longitude
              street_address
              city
              region
              postal_code
              iso_country_code
              parent_placekey
              location_name
              brands {
                brand_id
                brand_name
              }
              top_category
              sub_category
              naics_code
              phone_number
              open_hours
              category_tags
              opened_on
              closed_on
              tracking_closed_since
              geometry_type
                      }
                  }
              }
          }
      }
    }
  }
  ’)
  new <- Query$new()$query(‘link’, query)
  res <- conn$exec(new$link)
7

The json response doesn’t appear to be returning an end cursor

[1] "{\"data\":{\"search\":{\"places\":{\"results\":{\"edges\":[{\"node\":{\"safegraph_core\":{\"placekey\":\"222-222@63r-6bk-2x5\",\"latitude\":38.91756,\"longitude\":-76.964755,\"street_address\":\"2850 New York Ave NE\",\"city\":\"Washington\",\"region\":\"DC\",\"postal_code\":\"20002\",\"iso_country_code\":\"US\",\"parent_placekey\":null,\"location_name\":\"Washington Times The\",\"brands\":[],\"top_category\":null,\"sub_category\":null,\"naics_code\":3399,\"phone_number\":null,\"open_hours\":null,\"category_tags\":[],\"opened_on\":null,\"closed_on\":null,\"tracking_closed_since\":\"2019-07-01\",\"geometry_type\":\"POLYGON\"}}}]}}}},\"extensions\":{\"row_count\":1,\"version_date\":[\"safegraph_core: 1645948803__2022_02\",\"safegraph_geometry: 1645948803__2022_02\",\"safegraph_weekly_patterns2: 2022_01_11\",\"safegraph_monthly_patterns: 2022_01_11\"]}}"
8

Hello! Another problem we’re running into, is that we’re getting a bunch of rows back in the JSON that are all NULLS. Here’s the JSON response for five retailers in the 20002 zip area code:

[1] "{\"data\":{\"search\":{\"places\":{\"results\":{\"edges\":[{\"node\":{\"safegraph_core\":null}},{\"node\":{\"safegraph_core\":null}},{\"node\":{\"safegraph_core\":null}},{\"node\":{\"safegraph_core\":null}},{\"node\":{\"safegraph_core\":{\"placekey\":\"222-222@63r-6bk-4d9\",\"latitude\":38.917605,\"longitude\":-76.965407,\"street_address\":\"2800 New York Ave NE\",\"city\":\"Washington\",\"region\":\"DC\",\"postal_code\":\"20002\",\"iso_country_code\":\"US\",\"parent_placekey\":null,\"location_name\":\"Times Color Graphics\",\"brands\":[],\"top_category\":null,\"sub_category\":null,\"naics_code\":3399,\"phone_number\":\"+12026354035\",\"open_hours\":null,\"category_tags\":[],\"opened_on\":null,\"closed_on\":null,\"tracking_closed_since\":\"2019-07-01\",\"geometry_type\":\"POLYGON\"}}}]}}}},\"extensions\":{\"row_count\":1,\"version_date\":[\"safegraph_core: 1645948803__2022_02\",\"safegraph_geometry: 1645948803__2022_02\",\"safegraph_weekly_patterns2: 2022_01_11\",\"safegraph_monthly_patterns: 2022_01_11\"]}}"
9

@Sean_Mussenden_Univ_of_Md_College_of_Journalism it looks like the pageInfo { endCursor } isnt included in the query

10

that seems to have worked, got the end cursor to respond

11

@Aadit_Tambe can you try now? there was an issue with our updated release. issues should be resolved now.