Tool: Automated Dewey Data API Access

Your files on Dewey Data just became MUCH more accessible!

This notebook provides a tool to access Dewey Data in bulk via API. Simply follow the instructions to have your data saved into Google Drive, where you can continue analysis or download to your local machine.

Some useful features:

  • Download multiple data files in one go
  • Easily filter to the data products you plan to work with to avoid wasting eating up storage n your analysis environment
  • Preprocess your data before downloading into your analysis environment, allowing further storage optimization and saving compute power

While the File Browser in the Dewey Marketplace provides a handy interface to download your files, programmatic access may be preferable for many users. The structure of the requests shown in this notebook may be applied in other environments, such as in R on your local machine.

Run the code yourself and be sure to reply with any questions, feature suggestions, or other feedback!

1 Like

Everytime I click "Browse my files’, it shows ‘API Error’ and ‘Error retriving file list’. Do you know what is wrong?

Hi @yangsong , there may have been a temporary tech issue. Are you still seeing the issue?

Is there a list of updated colab tutorials like this? Google Colab

Hi @Yingjie_Li_MSU, the foot traffic provider has recently switched from SafeGraph to Advan going forward. We will be rolling out some Advan content this week (with Google Colab) so users can get a comparison of Advan vs SafeGraph.

In the meantime, I would suggest checking out this post, which discusses the methodology differences between Advan and SafeGraph. The post specifically touches on normalization for longitudinal analysis, the topic of the notebook you shared above. (Note, that notebook was made before there were pre-packaged normalized columns included in the dataset, so you may not need to compute the normalized visit counts yourself anymore if one of the included columns works for you.)