Add timestamp in data api?

Questions & Help
1

Hi, I want to get time (year, month) for the files downloaded from API, however, I couldn’t find useful information in the return of the API request to determine the time of a download file. For example, if I download a spend pattern file in SafeGraph, I don’t know which year it is in. Could you add an attribute in the API return to identify the time of each data file?

2 Likes
2

Hi Sangyx,

I have found the reference date for at least the movement patterns files embedded in the URL (i.e. “/2019-01-01/”).

Martin

1 Like
3

Add now I see that it is missing in some of the other URLs.

Ugh! I second this request!

1 Like
4

As of now, a subset of the most popular products have been date-partitioned (with a reference date in the URL). We’ll be shifting the remainder of products over to date-partitioned versions over the next week (where there is a relevant date in the data).

1 Like
5

Thanks for your help. Another question, What’s the difference between the files in one date-partition? For example:

https://amplifydata-production-dewey.s3.amazonaws.com/Spend%20Patterns%20-%20Entire%20US--eb6e748a-0fdd-4bc7-9dd7-bbed0890948d/2019-01-01/data_01ae7cd9-0004-b5a4-0043-0b8700d02d46_23_3_0.csv.gz

https://amplifydata-production-dewey.s3.amazonaws.com/Spend%20Patterns%20-%20Entire%20US--eb6e748a-0fdd-4bc7-9dd7-bbed0890948d/2019-01-01/data_01ae7cd9-0004-b5a4-0043-0b8700d02d46_23_3_1.csv.gz

Are they just one part of the whole data for this month?

6

They are just parts of the whole data for the month as you said - the data is split to keep the file sizes manageable under 250MB. The file names themselves don’t correspond to anything in the data.