Hi @ross_epstein_safegraph I have been using Unfolded.ai for the entire 2020. I noticed that there are some missing data in the “mobility data” in the summer period. They confirmed that there was an issue with the data transfer. Now, I am trying to see how I can get those data directly from safegraph but I don’t understand what package provides those data (for example “visits_by_day_total_Agriculture_Forestry_Fishing_and_Hunting”) at county level. These are supposed to be the “old” weekly patterns. Are these on catalog.safegraph.io? Thank you very much, really appreciate.
Hi @Nicola_Falco, all of the raw data is available on the catalog, but you will not find an aggregation like that – i.e. you will see the visits_by_day column, but you will need to filter down to Agriculture forestry etc and then again by county level (or vise versa)
You may have some luck in asking in the <#C012T1Z8FQA|unfolded-studio-support> channel whether a user or someone on the team has already broken this down manually and cut down on your work load.
Let me know if this answers your questions and/or if you have any follow up questions.
Hi @Jack_Lindsay_Kraken1 thanks for the answer. I actually contacted them first. I have been told that the weekly patterns were directly ingested and then aggregated geographically.
That sounds about right. Do you see where they got their aggregations from after looking at the schema?
Jack I am trying to look for the POI categories and subcategories but I cannot find the level of aggregation or code schema. Any insight? Thank you much for helping!
Ah i see. Are you familiar with NAICS codes? If i had to guess they used those to filter the Agriculture_Forestry_Fishing_and_Hunting
Code 11 appears to be agriculture forestry etc
County level will be done via FIPS/CBG (see attached photo)
Does that help?
Basically you cut the NAICS codes and the FIPS codes down to the level you need, filter accordingly, do your sum, then your data should be the same as the unfolded set
Jack, great! Thanks for the link. Now everything makes sense
Perfect! Keep me posted and feel free to follow up if you have any further questions!
Thank you very much for the support and help! I will definitely do.
Hi @Jack_Lindsay_Kraken1 hope you are doing well. I working on the data. So far I can decompose the FIPS, which is represented by poi_cbg, to get visit per county. To aggregate by “activity”, I am using the NAIC as you suggested. Now, to do that, because weekly patterns do not have NAIC code, I can use eaach safegraph_brand_id and link those to the NAIC using core places. The problem I see is that this would work for brands. I wondering what I could use to be sure to get the proper NAIC for each POI. Are safe_graph_place unique? Can you tell me if this strategy makes sense or if there is a easier way to link each poi_cbg to its NAIC? Thank you so much!
Hi @Nicola_Falco, think of weekly/monthly patterns as 1 set of data and core as another set of data –
once you know which POI you want (via their SafeGraph Place ID) you can merge on those IDs (i.e. the SGPID from core to the SGPID from patterns). They are unique and will never belong to another POI so you can rest easy knowing 100% they belong to the correct POI
Does this answer your question?
Yes, perfect. Thanks for confirming.
would recommend using Placekey as the ID - at some point the SafeGraph Place ID will be being deprecated
Thank you @ross_epstein_safegraph. I have a question about that. I got weekly patterns for summer 2020, downloaded from catalog recently. The data I have don’t have Placekey, Am I missing something?
you should be able to match those safegraph_place_ids to a more recent core places file that has placekeys
Oh I see, got it. Thanks