I requested geometry, places, and patterns data for March 2020 - September 2022 and I had some questions before diving in:
When I requested the data, I wasn’t able to filter to just the US. Is there a way of doing this without having to read each CSV file and filter them?
Related question: Is there an API or any way of querying the data I got? In general, I am trying to reduce the computing power needed for my research and an API would greatly help.
Finally, I am not sure how the geometry files change by date, especially since POIs are included even after they’ve closed. If I am interested in the area of POIs in a given state from March 2020 to September 2022, would it be sufficient to just look at the most recent geometry data from September 2022? Presumably, this file contains all the new places (some of which would be missing from, say, the March 2020 file) as well as old places that were open during the pandemic, even if they’re currently closed. Is this correct or am I misunderstanding the file system?
Hello @Sofia_Avila ! Sorry for the delayed reply. Here are some responses:
As of now, there is not a way to filter the data prior to downloading. This functionality may be available in the future, but there is no timeline. My recommendation is to use something like Google Colab to access the data in a cloud environment (via API – intro here) and filter as needed before moving the data to your analysis environment. We are hoping to build out a quick start Colab notebook in the near future to facilitate this.
I think I hit on this in the first bullet; there is an API that enables programmatic access to the data, but at this point there is no functionality to filter the data via API.
Your understanding is correct. Generally, the most recent geometry data is the most reliable. However, as you mentioned, they can change over time. The Places data contains opened_on, closed_on, and tracking_closed_since columns that may come in handy if your analysis wants to account for closures.
Please let me know if you have any follow up questions!
Hi @Sofia_Avila , sorry you’ve been having troubles with the API! I just shared a Google Colab notebook in another post to simplify the API process. All you need is your Dewey Marketplace email and password.