Does it mean you mostly capture data from mobiles on USA carrier plans?

Questions & Help
1

Thank you again for releasing beta-version of CAN (weekly) Patterns data and recently CAN-POI data. I was examining the carrier information in Patterns - about two thirds are non-null but there are NO Canadian carrier names that I recognize. Does it mean you mostly capture data from mobiles on USA carrier plans? My concern is about how representative this data is. Thank you!

2

Sorry, so only a small fraction of carrier-name fields are non-null (269k out of 290k in Dec), the rest being 4 or 5 big USA carriers. Can I assume than that NULLs are CAN carriers? My main worry is/was that if you only get data from USA carriers (or only smaller CAN ones) than the sample may not be representative. Thanks (tagging @Francisco_Utrera here as well)

3

Thanks so much for giving us a heads up about this, Rafal. Yes, you can assume that NULLs are non US carriers and there is no reason on our end to think that we only have smaller CAN carriers. I looked at the distribution by state and it was pretty representative compared to the true population, which gives me some confidence around representativeness.

Is there anything else other than the null values that make you think that the data might not be representative? I’m very interested if this is an issue

4

@Francisco_Utrera any updates on the home_panel summary for Canada Patterns? Would be great to have a way to normalize the patterns. Thanks!

5

no, but here’s how we recommend normalizing for CA:

  1. Remove outliers: POIs with null values or very few raw_visit_counts
  2. Scale raw_visits at the state level per week by dividing the state population (larger number) by our raw_counts (smaller number)
  3. Compute R^2 for each POI’s time-series separately and compute distribution statistics (mean, median, std, quartiles, etc.). In this case, it would be 12 points of data per POI since we have a 12-week sample.

Here’s a google sheet that does this for a specific location_name: How to normalize CA - Shoppers Drug Mart Case Study - Google Sheets

6

can you please try this and let me know what you think?

7

thanks, will give this a go

8

@Francisco_Utrera Nothing else yet - this started when I got a wrong (small) count of NULLs for carrier names and wasnt sure what to make of it (given closed border etc). Thank you!