Residing device count in 2023-07 and 2023-08 surged abnomorally

Hello Deweydata team,
I see the sum of residing devices in the home panel file surged in 2023-07 and 2023-08, even exceeding the US population. My current research needs a reasonable device sampling rate. Could you tell me how to address this issue?
Snipaste_2023-11-23_14-32-06

  • The sum of the “NUMBER_DEVICES_RESIDING” column is in millions.

Thanks,
Huan

@evan-barry-dewey I also facing the same issue on monthly data.
In addition, the POIs in 20230701 are spiked for the NACIS code starting with 62.

I am using the latest backfilled data.

Thanks @Huan_Ning_University_of_South_Carolina we’ll bring this back to the partner.

@Sled7424 can you provide more info on what spiked? Visits to POI? Number of POI? Etc.

@evan-barry-dewey
Number of POIs. If you filter POIs that NACIS code starts with 62 in monthly data, you will see the number of POIs is about three times compared to the number of POIs in 202306

Thanks, Evan! Hope they can correct the number soon!

@Sled7424 SafeGraph added a bunch of new Healthcare POI in their July Release. Advan uses the SafeGraph POI in their Patterns product.


We added over 3.1M POI in Healthcare and Social Assistance (the 62 NAICS family). :hospital::health_worker::stethoscope::tooth::nerd_face:

Here’s a breakdown of the largest subcategories:

  • +994,577 in Offices of Physicians (except Mental Health Specialists) (621111)
  • +993,679 in Offices of All Other Miscellaneous Health Practitioners (621399)
  • +713,942 in Offices of Physicians, Mental Health Specialists (621112)
  • +221,731 in Offices of Dentists (621210)
  • +87,477 in Offices of Chiropractors (621310)
  • +53,306 in Offices of Optometrists (621320)

@evan-barry-dewey
I merged the monthly data by month and filtered the POIs starting with 62, here are the results with the number of POIs:

20220101_US_62_1650708
20220201_US_62_1650708
20220301_US_62_1650708
20220401_US_62_1650708
20220501_US_62_1650708
20220601_US_62_1650708
20220701_US_62_1650708
20220801_US_62_1650708
20220901_US_62_1650708
20221001_US_62_1650708
20221101_US_62_1650708
20221201_US_62_1664980
20230101_US_62_1664608
20230201_US_62_1670706
20230301_US_62_1674112
20230401_US_62_1682457
20230501_US_62_1679388
20230601_US_62_1750738
20230701_US_62_4743650
20230801_US_62_1747222
20230901_US_62_1747222
20231001_US_62_1747222

In order to check the issue, I also looked at the weekly patterns data, but the weekly data looks good on my end:

20221031_US_62_1650708
20221107_US_62_1650708
20221114_US_62_1650708
20221121_US_62_1650708
20221128_US_62_1650708
20221205_US_62_1650708
20221212_US_62_1650708
20221219_US_62_1650708
20221226_US_62_1650708
20230102_US_62_1681458
20230109_US_62_1724186
20230116_US_62_1671332
20230123_US_62_1664608
20230130_US_62_1670706
20230206_US_62_1670706
20230213_US_62_1670706
20230220_US_62_1670706
20230227_US_62_1674112
20230306_US_62_1674112
20230313_US_62_1674112
20230320_US_62_1674112
20230327_US_62_1674112
20230403_US_62_1682457
20230410_US_62_1682457
20230417_US_62_1682457
20230424_US_62_1682457
20230501_US_62_1686181
20230508_US_62_1682704
20230515_US_62_1682704
20230522_US_62_1682704
20230529_US_62_1682704
20230605_US_62_1747222
20230612_US_62_1747222
20230619_US_62_1747222
20230626_US_62_1747222
20230703_US_62_1747222
20230710_US_62_1747222
20230717_US_62_1747222
20230724_US_62_1747222
20230731_US_62_1747222
20230807_US_62_1747222
20230814_US_62_1747222
20230821_US_62_1747222
20230828_US_62_1747222
20230904_US_62_1747222
20230911_US_62_1747222
20230918_US_62_1747222
20230925_US_62_1747222
20231002_US_62_1747222

Thanks for flagging. I’d recommend filtering the June/July data to the 1747222 POI that exist in the data the following months for now. I’ve reported this and it may be fixed in a future backfill but there is no ETA at the moment. I don’t know why Monthly was impacted but not Weekly. My guess is it had to do with when they received the new POIs from SafeGraph relative to when their Patterns release was going out. They may not have time to notice adjust before the July Monthly Patterns file was released.

Thanks! That’s what I planned for the next step. Just want to report the issue here.

1 Like

@evan-barry-dewey Hello, Evan! Has the data partner updated the residing device count in the home panel files for 2023-07 and 2023-08? Thanks!

Let me reach out to the partner for an update. I know we’re still waiting for a restatement of the home panel.

@evan-barry-dewey Hello Evan! Any updates for the July 2023 home panel data? I hope they have been corrected so that our team can restart the research. Thanks!

Advan just restated the Home Panel Summary files for 2023/24 Weekly and Monthly Patterns.

More info here: https://community.deweydata.io/t/advan-general-questions-support-and-change-log/26135

thanks, Evan!

Hello Evan! I saw a dramatic change in the residing device count in the restated Home Panel Summary files. Willl the monthly Neighborhood Patterns change? I have a study that relies on these two datasets. Such changes may heavily impact the results and interpretation.

Thanks!


Fig. Residing device changes before (2023-12-22) and after restatement (2024-01-19)

Hi @Huan_Ning_University_of_South_Carolina there is no planned restatement of neighborhood patterns to my knowledge. Is there a similar spike in the NP home panel summary?

Hi @evan-barry-dewey ,

Looks like advan only added 2018 new data, but the 2023 data is not updated.

Here are the Medical related record counts:

The number of POIs is highly variable.

Here are some weekly details comparison (number of POIs) using venn diagram :
20180101_vs_20230102_venn
20180101_vs_20230220_venn
20180101_vs_20230227_venn
20180101_vs_20230403_venn
20180101_vs_20230501_venn

@Sled7424 I believe more POIs have been added to the dataset over time. Is there a specific concern with this? I don’t think I understand the venn diagrams.

Hi @evan-barry-dewey,

Adding new POI is fine, but at the same time, Advan is also dropping POIs starting from 2023.

Let’s take the last Venn diagram as an example:

For the week starting from 20180101, there are 1,650,708 (16,717+1,633,991) total medical POIs (left red color plus the yellow part).

For the week starting from 20230501, there are 1,682,704 (1,633,991+48,173) total POIs ( the yellow part plus the green part on the right).

These two datasets have a total of the same 1,633,991 POIs, but compared to the 20180101 dataset, the 20230501 dataset has 16,717 POIs deleted and 48,713 POIs added.

The weekly datasets starting from 20180101 to 20221226 have the same number of POIs (1,650,708), POIs vary starting from 2023.

This likely occurred because Advan starting producing Patterns at the start of 2023. At that time, they likely just did a backfill of Patterns data to POI as they were in that point of time. Moving forward from that date, POI are dynamic and might change as locations open and close. So it makes sense the POI are added and dropped after 2023 but not before.