Hello! What is the date range for a POI in the Core Places data? For example, does the March 2020 file include POIs that were identified in March 2020 alone or does it include POIs identified over the previous 12 months (i.e., March 2019-March 2020)?
Hi @Alex_Zentefis_Yale_University, yes and no. Unfortunately the answers to these kinds of questions are never as straight forward as one might hope
The most recent version of Core will have the POI that were used that month (not necessarily identified that month - but basically a full list of certifiable POI). From Core release to Core release the POI change, and you can see/track those changes in the Change Log.
With that being said, the majority of the Core POI dataset will remain the same from version to version, but if you want 0 chance of missing any POI data, you will need to use the Core corresponding to each patterns data month you are using.
We do, however, recommend users to use the most recent version of Core if at all possible to hopefully have the highest quality data (with the most bug fixes)
I hope this helps answer your questions, please let me know if any part of this was unclear.
Hi Jack, thanks a lot. This is extremely helpful!
Could you elaborate on what you mean when you say:
“…but if you want 0 chance of missing any POI data, you will need to use the Core corresponding to each patterns data month you are using.”?
Sure @Alex_Zentefis_Yale_University, so essentially, if you look through the aforementioned change log, you will see some POI were dropped for 1 reason or another - maybe they closed or maybe they were miss labeled etc. The patterns files for those time periods will still have those POI in them, so when you try to merge on the most current version of Core POI you may not be able to find those specific SafeGraph Place IDs (because they were dropped in the past) - does that make sense?
Yes, that makes sense @Jack_Lindsay_Kraken1. Does the Patterns backfill help with that issue, though?
Absolutely! But backfills only come every so often (about once ever 6 months). The reason I mentioned the core data from past months is in case you start your analysis today and 4 months from now, you need the data in between and there is no backfill yet.
I gotcha. Thanks a lot. Is the latest Core the 1/6/21 version then?
That is correct, though I believe the new Core (for Feb) will be coming out in the next few days (within the first few days of each month - usually)
Okay thanks! For future questions, should I return to this thread or pose a new question to #safegraphdata?
I always recommend a new question in the #safegraphdata thread - it improves the visibility of the question!
Okay. Sorry just realized this - if we wanted to line up the Patterns data with the Core, and we were, say, looking at Sept 2019 Patterns, which Core should we use?
a few things to knock off the checklist:
- if it is backfill, take the Core that used to create the back fill
- if it is the original, for 100% coverage, use the Core from that period
- If it is the original and you want the most accurate data (least buggy), use the current Core data - expect to not have a 100% match rate though
Am I right that the oldest Core still available is March 2020? That’s the latest back I see available in my account.
Looks that way for me too – so in that case, it should be the backfill (I assume the 2019 data you are looking at is in a backfill, is that correct?)
I was looking at “Monthly Places Patterns (aka “Patterns”) Jan 2018 - Apr 2020”
Is this the backfill?
link you to*
Happy to help! Please let me know if I can help with anything else!
Hi @Jack_Lindsay_Kraken1, I was re-reading what you wrote and just want to clarify something. You said that “…you will see some POI were dropped for 1 reason or another - maybe they closed or maybe they were miss labeled.” Are permanently closed places dropped from the Core updates? My understanding was that permanently closed POIs are still available in Core (and identified using the open/close columns).