I’m trying to sync monthly and weekly patterns backfill data through s3 and having some issues. Can anyone help?

Questions & Help
1

I’m trying to sync monthly and weekly patterns backfill data through s3 (my team’s integration is built around s3, not the shop data), and the folder structure is confusing to say the least. For weekly patterns, I’m using weekly-patterns-delivery-2020-12-backfill/patterns_backfill/2020/14/21/... and from there there are folders for 2018, 2019, and 2020, which seem to make sense.

For monthly places patterns, I can’t tell what I should use. My best guess is it should be monthly-patterns-2020-12/patterns_backfill/... but from there I’m not sure since there seems to be a number of recursively nested folders. As best I can tell, something probably went wrong when Safegraph was syncing data to s3, and it ended up incorrectly syncing to subfolders numerous times (something I’ve managed to do in my own projects before).

Any help would be much appreciated.

This topic was automatically generated from Slack. You can find the original thread here.

2

For monthly patterns from the July 2021 release and onwards, you’ll want to use:

s3://sg-c19-response/monthly-patterns-2020-12/release-2021-07/

For monthly historical patterns, you’ll want to use:

s3://sg-c19-response/monthly-patterns-2020-12/release-2021-07/*_backfill/

3

Thanks so much.

4

No problem! To prevent any further questions from being overlooked, I’ll go ahead and close this thread out. If you have any more questions or follow-up questions, we’re always here to help! Just be sure to make a new post to help, as we aren’t monitoring old threads at this time. Thanks!