I have a question about which core places version we should use for the “Weekly Places Patterns Backfill for Dec 2020 and Onward Release”. the documentation seems to imply that we should use the “dec 2020 release of Places” for all weeks in the backfill, and the release metadata says “11-2020” for the core_places_version_used. So I am guessing the version of places I should actually use is from “Core Places US (Nov 2020 - Present)”, and use the version which corresponds to 11-2020, ie at path core_poi/2020/11/06/12 - is this correct?
Hi @Emma_Pierson_Stanford, can you please point me to the documentation where it implies we should use “dec 2020 release of Places” and also where the release metadata says “11-2020” for the core_places_version_used? We may need to update the documentation if it’s ambiguous
sure! one sec.
the release metadata i downloaded came with the Weekly Places Patterns Backfill for Dec 2020 and Onward Release. i just spot checked a couple files, and it said “11-2020” for the core_places_version_used.
the documentation i’m looking at (FAQs | SafeGraph Docs) says “Activity from January 2018 through and including December 2020 was generated using the Dec 2020 release of Places. This is the first historical delivery that considers point-in-time POI openings/closures. For example, if a POI opened in January 2019, we will not attribute visits to the POI from January 2018 - December 2018 and will only attribute visits from January 2019 onward. On the other hand, if a POI closed in January 2019, we will only attribute visits from January 2018 - December 2018 and will not attribute visits from January 2019 - present.”
could you clarify which version of core places i shoudl be using to correctly match with “Weekly Places Patterns Backfill for Dec 2020 and Onward Release”?
Thanks! Okay, so this is from the Weekly Pattern Docs:
> 5) We produce our Core Places file monthly and start using the new file for our visits generation on the first of each calendar month. This means that if we introduce a new place on the 1st of a month and the Weekly Patterns file straddles two months, you will only see visits in those days of the week that are in the new month. This should only show up in a fraction of places each month.
I would suggest using the most recent month’s Core Places. Places includes permanently closed POIs, so the most recent release should have everything from previous releases. In truth, Places usually will not change a ton from one month to the next. However, if you find that some of the POIs you want to analyze are showing up in Patterns but not Places, please make another post about that (and feel free to tag me in it)!
Does that make sense/help?
thanks! “I would suggest using the most recent month’s Core Places” ~ so to be very concrete, we’re trying to analyze data from jan 2018 - dec 2019.
it seems like we want to analyze match with the core places file that was actually used to generate the weekly patterns data in “Weekly Places Patterns Backfill for Dec 2020 and Onward Release” in that timeframe. my sense was that that was the 11-2020 data - is that not true?
it seems to me that it would be preferable to a) use a single core places for all weeks in the patterns data, and b) use the core places that was actually used to generate the patterns, which i would hope be recorded in the core_places_version_used field. but maybe that is wrong.
Hi @Emma_Pierson_Stanford , the FAQ reference is to our Patterns (a monthly patterns calculation) product, not our Weekly Patterns product. You should be able to trust the core_places_version_used and move forward with Nov 2020 Core Places.
Sorry for the confusion on this!
perfect, thank you. we will confirm that core_places_version_used is indeed consistent across all the weeks we intend to use, but hopefully it will just be nov 2020 and we can use that.
thank you for your help!
Great, you’re welcome! Please do let me know if Nov 2020 is not consistent across all weeks you use, or if any issues come up!
will do.
confirming: core_places_version_used field is always 2020-11 in the metadata corresponding to each week from 2018 - 2019.
however, my student reports that 5% of the places in the weekly patterns data from 2019.1.7 are missing from the core places data
do you know why this might be?
@Emma_Pierson_Stanford That’s interesting… I’m really not sure why that would be the case, but I will look into it further. A couple questions:
- Have you checked this for any of the other weeks?
- Are there any patterns you notice about the missing POIs? (many of a similar type? many in the same geography?)
- Are you using the entire Core Places data set and the entire Weekly Patterns for that week? (did you filter by region/NAICS/etc?)