Components
/
Bulk Import

Bulk Import

Upload a CSV/TSV file and use its data to bulk execute any function, including updates and inserts (as well as any custom functions you’ve created). The bulk import component can provide an easy way for users to update or insert new data records en masse.

Ways to use a bulk import

Update many records at once

If you have a file with updated data for a number of existing records, you can use bulk import to update all of the records instantaneously.

Example: You sell gadgets online and there’s a newer version of the products that you carry. You store your product information, like the product name, price, and description, in your company’s database. You can use bulk import to update all of the new product descriptions, without changing other existing data points (such as the price or name of the product, which did not need to be updated).


Insert many new records 

Create a series of new records by uploading the required data for a new record.

Example: You’re an on-demand scooter-sharing service and you just bought a new fleet of scooters. You want to import all of the new data (like the scooter serial number and firmware version) as new Scooter records in your database.


Configuring your bulk import

Check out our tutorial on using the bulk import component to bulk update records: 

Actions


First, choose the function that you want to be executed with the data that you import. You can choose to insert/update/delete data, and you can also use this to bulk perform any custom function you want. 


For most databases, you'll see auto-generated insert, update, and delete functions. For example, if you selected your PostgreSQL database with a Users table, you'll see "Insert Users record", "Update Users record", and "Delete Users record" functions appear automatically.

For most business apps, auto-generated functions will appear, based on the available APIs of that app. For an app like Stripe, you'll see functions for "insert Balance Transactions record", "update Balance Transactions record", "delete Balance Transactions record", and so forth for the resources available in Stripe's API.

If you created any custom functions for a database or HTTP service, those will also be available for you to choose from.


Refreshing components after submitting

Usually, if your import changes some data that is‍ displayed in the Space, you'll want to refresh the data. On Submit options allow you to refresh the data of any component in the Space.


Fields

Here you'll specify the values that are used when the bulk import occurs. To do this, you’ll select fields, and then specify what values go in them.

Any required fields will be shown and you must specify where the value comes from. 

  • For updating or deleting records, the primary key is a required field. 
  • For inserting new records, any field that is required for a new record will also appear. 

Click +Add Data to add any non-required fields that you wish to edit. Click “Edit” next to each field to configure how the value is determined. 

For the bulk import component, you’ll see an option in the dropdown to “Use a value from a CSV or TSV.” This will allow you to use values coming from the bulk import. 

You can also choose to set a custom value, get the value from another component, set a UUID, or set to null - see Field Configuration to learn all the options.

In most cases, you’ll want to use a mix of these configuration options to create an ideal bulk import experience. Let’s go through an example, where we have a file of new customers that we want to bulk import. 

First, we’ll select the “Insert company record” function under Actions, this will add the new data to the Company resource within our Postgres data source. 




Then, we’ll configure the fields we want in this way: 

For the id field, since these are brand-new Company records, we’ll have Internal generate a UUID for each record. 


For the create_at field, we’ll set this to the date and time the button is clicked - this means that the time in which a user imports the file will be the time recorded in this field. 

For the company_name, we’ll want to import this from our file, so we’ll choose “Use a value from a CSV or TSV”. 

 

And we’ll also take the phone_number field as well from the imported file. In this case though, we’ll let this field be blank, as not every Company record has a phone number listed. 


Once you’re done, save your bulk import component and publish your space so it can be used. 

Using your bulk import 

When a user clicks on the bulk import button, they’ll be prompted to upload the file containing the data. Internal accepts both comma-separated-values (CSV) and tab-separated-values (TSV) file types. 

Once you upload your file, you’ll be able to map the columns in your field to the fields in your resource, using the dropdown selection at the top of each column. Unmapped columns will not have their data imported. 


Next, you’ll review what your data looks like before importing it. Internal will alert you if there are any potential errors, such as data that doesn’t match the field’s filetype. For these types of errors, you may still proceed to import the data, although it may fail. 

You may also see blocking errors that must be resolved before you can import. For example, if you have an updated_at column that is required, if your file has any rows where this column is blank, you won’t be able to proceed to import.  


Once you’re ready, hit the “Import” button. 


Note: If there are failed imports, Internal will provide a CSV of all the rows that did not import.