It might be very basic, but can anyone suggest me, how to access the data (as my command prompt is showing that the dataset is downloaded, but i’m not sure how to access the data)?

General Discussion (Archived alternative-datasets
1

it might be very basic, but can anyone suggest me, how to access the data (as my command prompt is showing that the dataset is downloaded, but i’m not sure how to access the data)

2

Hi Shakil,

by default you tell it to store the data in your local directory. more than likely that means it is under <c://users>:{your name}/file.csv

however, the easiest way to locate it would be to open your file explorer and type in the full file name of one of the CSVs downloaded and right click it and click ‘open file location’ – you can then move them from there anywhere you like, just remember when you redownload the data it will refresh in the original spot.

I personally take where it saves the data and bind it to my quick access bar on the left.

if you are on a mac, let me know, the steps will be slightly different.

Let me know if this helps!

3

Thank you Jack. I was able to find the zipped dataset. My next questions is, I was trying to open the dataset after extracting from the zip file. Unfortunately, I was unable to open it (2020-04-05 weekly pattern corrected) using different software (i.e. r-studio, mysql, excel). Can you guide me regarding that? Thanks in advance !

4

Sure! depending on the size of the dataset, it might crash something like excel. mySQL should be able to handle it, but again it just depends. I know when I moved it into a datafame in pandas I had to adjust my heapmemory in my IDE and process it in chunks of 1000000.

To better understand your system, could you let me know how much RAM you are working with, and also how many gigs is the file you are trying to access?

5

also, here is a great article on the multiple methods for extracting large datasets. The general rule of thumb is only take what you need. For instance you can pull the first 100 rows of every column to get a better idea of what exactly you need then custom pull those columns from the csv.

I hope this link helps: Open Big CSV Files - CSV Explorer Blog

6

@Shakil_TTU I run into the same issue when tried to open it with Excel. I ended up using SAS and it worked quite well.

7

in R, if they are too large to read into memory via data.table::fread you can use {disk.frame} Larger-than-RAM Disk-Based Data Manipulation Framework • disk.frame or store them in a SQLite database with {RSQLite}

8

Thanks Jack, Tian and Joe :slight_smile: