Can I know how to get the weekly mobility data (or social distance) data by state level?

Help
#1

Can I know how to get the weekly mobility data( or social distance) data by state level? thanks

#2

@QIULI_SU_Louisiana_State_Univerisity

the data is distributed at the level of census block group.

the first 2 digits of teh CBG are the State FIPS

so you can re-aggregate based on that to get State level summaries.

#3

But I download it sucussfully, but don’t know how to use it and how to aggregate to state level?

#4

I Mean I download" aws s3 sync s3://sg-c19-response/social-distancing/v1/ ./weekly-patterns/ --profile safegraph"

#5

but don’t know how to use it and how to aggregate to state level?

#6

to measure the daily and weekly mobility

#7

Thanks, but still don’t know:1) the data “aws s3 sync s3://sg-c19-response/social-distancing/v1/ .” is by day, is the aggragted data from Jan 1st til today daily? 2) how to join the Census Data with aggrated daily dataset of (aws s3 sync s3://sg-c19-response/social-distancing/v1/ )? thanks

#8

@Ryan_Fox_Squire_SafeGraph Thanks, but still don’t know:1) the data “aws s3 sync s3://sg-c19-response/social-distancing/v1/ .” is by day, is the aggragted data from Jan 1st til today daily? 2) how to join the Census Data with aggrated daily dataset of (aws s3 sync s3://sg-c19-response/social-distancing/v1/ )? thanks

#9

@QIULI_SU_Louisiana_State_Univerisity what analysis tools are you using for working with the data?

#10

@Ryan_Fox_Squire_SafeGraph I use R and python jupyter notebook.

#11

@Ryan_Fox_Squire_SafeGraph thank you very much!

#12

@QIULI_SU_Louisiana_State_Univerisity OK so what are you trying to do, what commands are you using, and what error are you seeing? I am not sure exactly what you are trying to do.

#13

@Thank you very much for your reply. what I want to do is : step 1) combine the all daily social distance file from the data “aws s3 sync s3://sg-c19-response/social-distancing/v1/ .” into one aggraged file 1. Step 2) joined file 1 with Census Data in order to get another aggrgated file 2 with state level daily spocial distance information and sensus information. But I don’t how to do both steps.

#14

1st download all the files to somewhere on local computer (e.g., your desktop) using the aws s3 sync command

did you do that?

#15

Yes. I download the file with command"aws s3 sync s3://sg-c19-response/social-distancing/v1/ " and I should manually combiend files one by one?

#16

i mean I only can complete step 1 manually?

#17

There are ways to read the data into python or R directly from S3 without manually downloading it, but manually downloading it is the easiest. There are also ways to read all the files into one object in Python without having to manually combine the files. I would not manually combine the files.

Are you using pandas? You can read each file into pandas and pd.concat into a single dataframe

#18

Cc: @Yang_Li_Insight_Data_Science

#19

@Ryan_Fox_Squire_SafeGraph @Yang_Li_Insight_Data_Science I use command " aws s3 sync s3://sg-c19-response/social-distancing/v1/" download the file already. Since the daily file is seperate folder, if I want to exam daily change, I need a file include all daily data. I can use pandas command. Do you have some sample code ? thank you very much.

#20

Are you trying to 1) combine the two datasets and then 2) aggregate to state level?