How to Import Data into Airtable without Duplications

Recently, I've received several questions about importing data into your Airtable database. Before I just jump right in and start discussing the how - let me first address some definitions and scope questions.

Firstly, when I think about IMPORTING data, I instantly imagine that we are discussing a bulk data transfer. If you have questions about bringing individual records into a database, those are answers you won't find here... Typically individual records (entries) are handled via form submissions, or automated procedures (i.e. someone pays an invoice and an automation marks the record as "Paid" in the database). So, for the purposes of this post, we are speaking about bringing in a large amount of data, probably somewhere north of 50 records.

Secondly, when we import data into Airtable, we are taking it from a 2D environment (spreadsheet, CSV file, etc.) and we're moving into a 3D environment (relational database)! This means that we are only able to import data to one layer, or table, at a time. This can be frustrating and time intensive, but you can still link to other tables during your import if you are able to match the name of other records.

Now that the groundwork is laid, we can discuss the actual data import process. There are two options to consider:

1. Copy + Paste data from a spreadsheet into your table
2. Use the CSV Import Block

Option #1 is quick and relatively painless for small data sets, but Option #2 has a ton more functionality and is the preferable choice for substantial data imports. Of course, this requires that you are on a 'pro' Airtable account in order to unlock the Airtable Blocks.

In this video I outline the mapping process with the CSV Block, but more importantly walk through the steps to "Merge" data with your existing records. This is critical in minimizing data duplication. In this example, I show how 50 records are condensed to only 40 after we ask the block to merge data based off the email address.

In most cases, importing data into a database will only happen one time - from there the database should update as your operations progress. Still, its advisable to keep your Importing skills sharp, as you never know when you'll need to move around a bulk amount of data!

_________________________________________

Learn more at these links: 🔗 👇

Join us for our next live training!

Take our free Airtable Crash Course!

Getting Stuck? See how we can help!

Get our Airtable Field Type download!

_________________________________________

Table of Contents: 📖 👇

00:00 - Intro
00:39 - Outlining the Topic
02:11 - Field Types (Dependent vs. Independent)
05:18 - Organizing Fields for Copy/Paste
06:55 - Matching your Data Type & Format
09:20 - Performing a Copy/Paste
10:01 - CSV Import Block
12:59 - Merging Records to Avoid Duplicates
14:59 - Get More Help

_________________________________________

Software Resources (may contain affiliate links): 💻 👇

_________________________________________

#GAPConsulting #GarethPronovost #Airtable

[JohnEndres]

Hi Gareth, a vote of thanks from Johannesburg, South Africa. Really appreciate your videos and clear explanations. Here’s our use case for importing data without duplicates: we want to track our quote status in Airtable. To do so we export the list of current quotes from our quoting software and then import it into Airtable. Being able to do so without duplicates is a huge help and I can’t wait to try it tomorrow. Thanks!

[janicewilliams]

Excellent, simple explanation. Whew. Thank you.

[VaskenBakalian]

Awesome detailed tips like the datetime format (European vs US) !!

[mabo]

Great info, as usual, Gareth. Quick question, what software do you use to edit your videos?

[Ed_Gein]

also, we want to create the database and later add a new field to add to the end of every record in the database

[tayoamos]

This was super helpful! Quick question...what do you do when you want to have multiple items in one field? For instance, I want to put in a directors info and their agents. But sometime they have more than one agent. This information is in the .csv but not sure how to import...do I have to manually add/ edit this information? Thanks!

[arellaflur]

Hi Gareth, This is a helpful video. I am in a situation where I have monthly CSV records that I want to upload to AirTable without duplicates. My one further question is does AirTable have a way to recognize if specific fields have been updated? For example, if I already have last month's member data uploaded and I am going to upload the new month's, does AirTable have a way to recognize that someone who is already in my records from last month has, say, updated their email address? So I would want to remove the duplicate record but have the email field be updated. Thanks!

[samanthak]

Gareth - Your training and course is amazing. I thought this example might work but not quite what is needed. I am stuck on trying to find an App that will remove the OLDEST duplicate data fields after I upload new data without going through one by one. (I am adding to an airtable removals from Az and when the status changes to complete I want to remove the older data line that says pending.) I can figure out how to do it one by one or remove the newest line...but not the oldest. Any suggestions? Thank you so much!!

[federicoscolari]

Hey Gareth, great video as always. Does this apply to the entire table regardless of the view? I'd like to repopulate only a portion of my records, so I filtered out the ones that I want to keep untouched, and imported / updated the remaining part. I ran some tests but it still isn't clear. Thank you!

[haf]

Hi Gareth, I receive a csv file each week, which I want to store / update on google sheets, what is the best way to automate this process? I have looked at building a zap, but since it is a massive csv file with 1000+ rows and 15+ columns, I wasn't able to build a zap for this. Thanks for your help!

[yogeshsharma]

Thanks for sharing this Gareth.
I'm using webhook to send data from my website form to Airtable, but in some cases the same person fills that form for different purpose and it's creating multiple records in Airtable for that person. And I only want to keep one record per person and merge few fields of the new record and update in existing Airtable record. Is that possible?

[Ed_Gein]

How would you import a field of attachments or images? It seems like a simple import of an excel spreadsheet with images does not transfer unless i am not matching the field name/format

[arifmuhammad]

can you trigger the csv import from a button and have the file upload in the background assuming you already have the mapping created once?

[kevinleesmith]

What's the difference between a block and an app??