How to Read Multiple CSV Files with For-Loop in R

Got TONS OF CSV FILES? Want them all consolidated? Here's how to read multiple CSV files with R using for-loops and with purrr map(). Here are the important links to get set up 👇

Once you take these actions, you'll be set up to receive R-Tips with Code every week. =)

THE R-TRACK PROGRAM
==============================
I have a complete system designed to teach you R, Data Science, Advanced Machine Learning, Web App Development, Time Series, & more - all for business. It's insanely powerful. Over 6+ months, learn everything needed to become an R Rockstar for Your Organization.

[naomihuntley]

Thank you so much for taking the time to make this video. You did an amazing job of breaking this all down so that I could understand each step!

[naomihuntley]

Thank you so much for taking the time to make this video. You did an amazing job of breaking this all down so I could easily understand :)

[stephenlung]

Following up from this, in order to add each list item into the environment variable as a tibble. R Code
:
list_of_file <- file_paths %>%
map(function (path) {
read_csv(path)
})

# convert each list into a tibble
list_of_file%>% map(as_tibble) %>%

# set each list item to an environment variable
list2env(envir = .GlobalEnv)

[wbdill]

some of the files read "drv" as a logical and some as a char. You can specify column types in read_csv with col_types. Also data.table has rbindlist which will combine all the data frames in each list element into a single data frame.

lst_2 <- file_paths %>%
map(function (path) {
read_csv(path, col_types = "ccniiccnncc")
})

df_all <- data.table::rbindlist(file_contents, idcol = FALSE)

Very well explained and easy to follow. Thanks! ❤❤❤

[chibuzoriluno]

So how do I view each individual file after using the map function?

[sanketdeorukhakar]

How will you convert that list into different data tables

[soxfan]

Thanks for this, it was really helpful. If the CSVs i was mass importing were time series data, but the CSVs didn't have columns with time data, can i import csvs and add the column(s) with time data (taken from file name) in the same loop, or do I need two separate loops?

[sanahusain]

file_paths is not working for me . It's giving the error ' failed to search directory ' . I am typing the name of the folder where my .csv files are saved. please help . thanks

[mkklindhardt]

First of all thank you!

Thank you so much! I have >100 RData files..
Could I use this approach to import/load multiple RData files (each with single data.frame objects) into my R environment? And finally how to merge all these (RData) data.framees into one single dataframe?

[nilosmitabanerjee]

Thank you so much for such a useful video! Definitely signing up for the news letter :) !
Just one question - when I try the purr method I get this error: Error in as_mapper(.f, ...) : argument ".f" is missing, with no default. Can you help?

[RobisK]

Can this be adapted to data stored in multiple subfolders within a main folder?

[mrgomelonsolaris]

file_paths %>% readr::read_csv() should be enough, because read_csv understands multiple filepaths and row binds the data.

[danhdo]

Thank you! Just a follow up question, how do I start working with these data frame? For example, I'd like to View() a file. What should I put in the brackets? :( Sorry if this is a silly question. I could re-create what you did, but then I don't know to do next.

[musicalmbh]

Thank you for this post, is very good!
Please, how can I proceed to work with 2 columns of each of my 35 .csv files imported? I want to compare the area changes of the categories in the files, for example. Thank you so much in advance!

[quentinmcteer]

In the for loop, I would like to use some other arguments in the read_csv function (skip and col_types) because my csv's are messy. When I try to include those arguments directly in the for loop (skip = 3, col_types = cols()), they don't seem to run, any ideas? Here is my code:
for (i in seq_along(file_paths)) {
file_contents[[i]] <- read_csv(
file = file_paths[[i]],
skip = 4,
col_types = cols())
}

[jdcarter]

i got it to run without any errors but i wish i could place each csv in a variable so it can appear in the environment window instead of just appearing as a tibble

[MrCeroSombra]

I've been trying to work through a project in which I need to use this very procedure, however it repeatedly gives me the error that object file-paths does not exist.

[qmbikerider]

I need to import a large file, perform a series of steps, output to a .csv, and then import the next file from the list of files and repeat (for 139 files). Is this something I can tweak from what is here, or should I go about it entirely differently? Thanks in advance for any help from anyone who has done this.

[khoale]

how do you combine all those data into one dataframe. Can someone please help