What is the difference between the files in weekly-patterns-delivery/weekly/normalization_stats_backfill/2020/12/07/20/ and the files in weekly-patterns-delivery/weekly/normalization_stats_backfill/2020/12/07/21/?

What is the difference between the files in weekly-patterns-delivery/weekly/normalization_stats_backfill/2020/12/07/20/ and the files in weekly-patterns-delivery/weekly/normalization_stats_backfill/2020/12/07/21/ ? They seem to be the same, but both are in the AWS bucket. Are they different? Is one preferred? Thank you!

Hi there Nick! Sorry for the confusion. If you resync one more time, the latest version is the fully correct version, and I have deleted the incorrect backfill versions from the bucket. It should be the path weekly-patterns-delivery/weekly/{name}_backfill/2020/12/14/21/{yyyy}/{mm}/{dd}

Still some issues here? The full weekly patterns dataset is almost 1TB right now. Seeing the following folders (each about 207GB) in the AWS bucket

/patterns_backfill/2020/12/07/20
/patterns_backfill/2020/12/07/21
/patterns_backfill/2020/12/14/19
/patterns_backfill/2020/12/14/21

Let me double check that it’s been synced correctly :slightly_smiling_face:

Same here. 2020-12-07-20 and 2020-12-07-21 have exactly similar files and folder structure. Just checked.

Appears to be updated!.. now if I could just get the --delete flag to work in my sync command… :confused:

Hey folks sorry I’m having trouble following this … so I have the patterns folder and the patterns_backfill folder. The latter has through 11/30 2020 but patterns itself starts with 6/24/2020 … which should I be using?

Hi Andrea!

The patterns_backfill folder will have the latest data.

Okay so ONLY use patterns instead of patterns_backfill for December (which isn’t in backfill )?

Yeah, when the December monthly patterns gets delivered it’ll be in the patterns folder (and, of course, the weekly exists in the weekly delivery folder and should have up through 12/13 right now :slightly_smiling_face:)

Okay thanks!