We’re just getting started analyzing the *social distancing metrics* and have a few questions!

Questions & Help
1

We’re just getting started analyzing the social distancing metrics and have a few questions!

(1) We are wondering if we should be scaling the metrics using the candidate_device_count over the device_count, and if that should apply to all social distancing metrics. This question follows directly from a conversation started January 19th 2021 between @Ryan_Kruse_MN_State, @Aaron_Flaaen_Federal_Reserve_Board, and @Francisco_Utrera, which I followed the thread for, but wanted to know if there were any takeaways regarding which variable works best for scaling.

(2) We are interested in getting indices of change for social distancing metrics from a pre-pandemic baseline and are wondering what others’ experiences have been as to the ‘best’ way to define this baseline?
I’ve seen this done as a difference or ratio from a single week in early 2020, e.g. the week of March 1, 2020, before any interventions were in place.
I’ve also seen this done as a difference or ratio from a similar day in the previous year, e.g. if we’re getting the change from a baseline for a metric on 10-24-2020, a weekday, we’d use as the baseline the average weekday metric for weekdays in October 2019.

(3) Due to the data product change to v2.0 during Jan1-May 9, 2020, should we assume that we cannot get indices of change for data during this time period, if defining the baseline as coming from a similar day in 2019? That is, is it not possible to create indices of change that compare data coming from v2.0 and from v2.1 time periods?

I realize this is a lot of questions — answers to any of the above would be helpful! Thanks in advance.

2
  1. I haven’t used candidate_device_count myself so I can’t really say on this one, hopefully one of the people you tagged will pop in.
  2. This really comes down to the question you’re trying to answer. If you’re trying to look at an overall time trend of at-home behavior, going to a single week like March 1 makes a lot of sense, as it ensures everything is comparable (outside of v2/v2.1 issues). However, if for a given time period you want to know how that time period is relative to a baseline, comparing to that same time period (or a rolling seasonal average) one year before is more appropriate.
  3. Backfill for this date range isn’t available and so comparisons to data outside that range are going to be a little spotty, although internal comparisons should be fine. You can try to make things more comparable using statistical adjustments (say, scaling upwards in an attempt to get May 1-9 to match May 10-18) but the data collection procedures are just not the same, comparisons aren’t going to be clean here.
3

Hi @Abigail_Horn_USC

  1. Perhaps @Aaron_Flaaen_Federal_Reserve_Board has some suggestions here. Based on the thread you referenced, Aaron was going to try scaling by candidate_device_count , so he may have some insight.
  2. (and 3) I agree with Nick, and have one other suggestion. You might be able to get around some of the v2/v2.1 issues by looking at the relative changes. For example, how do the changes from March 2019 to April 2019 (all v2.1) compare to March 2020 to April 2020 (all v2.0)? I can’t make any promises about how well this approach would work though.
4

@Nick_H-K_Seattle_University and @Ryan_Kruse_MN_State, really helpful answers, many thanks. I can now think conceptually about the appropriateness of each kind of baseline (and am most interested in the overall time trend of at-home behavior so will use the week of March 1st baseline).

We’ll explore making scaling adjustments to the v2.0 data to make it comparable with v2.1. Something else that just came to mind is using week of March 1 2020 as the baseline for data from March 2 - May 9, 2020, and then using some date or rolling average from 2019 as a baseline for data from May 10 2020 - current times. That way v2.0 is always being compared with v2.0 and v2.1 with v2.1. (Ryan, maybe that was what you were suggesting?) Anyways we’ll explore and if we find anything interesting will report back here!

Thanks for the guidance!

5

@Abigail_Horn_USC You’re welcome! Yes, that is the idea I was trying to get at Please do let us know how it goes so we can get a better idea of what works and what doesn’t work!

6

Will do, thank you!