When Do you Use Salesforce Bulk API?
Bulk API is actually based on the REST principles and it is set up to successfully load or delete large amounts of data. It can be used to asynchronously update, insert, query, delete large records by submitting them in batches which are then processed behind the scenes by Salesforce. Bulk API also makes it easy when it comes to processing data in the form of records ranging from thousands to millions. However, in order to avoid writing a new client application, simply enable your Bulk API to process records in the Data Loader by use of CSV.Working With Bulk API
Bulk API allows the user to asynchronously query, update, upsert, insert or simply delete a large amount of records. Binary attachments like Attachments can also be part of the records.When the HTTP POST call is used, you will have to send several batches to the main server, which will in turn process the batches. As the batches undergo processing in the background the user is able to track the progress by simply checking the status by use of the HTTP GET call. All the operation always use the HTTP POST or GET procedure to send or receive CSV or XML data.
How Does Bulk API Work?
A number of records has to be processed by simply the introduction of a job that has one or several batches. The job has to specify the object being processes and the kind of action that is used either query, upsert, insert, update or delete.The records sent through the HTTP POST request to the server is known as a Batch. Every batch that is independently processed by the server and it doesn’t have to be processed in the order in which it was received. The client has the option to choose the best way to separate the larger number of data into the desired number of batches.
A job is usually represented by jobInfo resource which helps in the creation of any new jobs, get the status for existing ones and changing the job status when necessary. A batch is usually created by the submission of an XML or CSV representation of several records along with any existing references to the binary attachments in the HTTP POST request.
After it is created the batch status can be consulted through the BatchInfo resource. When the batch successfully completes, the results for every record is always available in the result set resource.
Processing data actually consists of the steps outlined below:
- Introduce a new job and specify the action and the object in your creation
- Send the date to serve in several batches
- After all the data has been successfully submitted, simply close the job. Note that after you close the job, you can’t send any more batches as part of the initially created job.
- Check the status of all the batches at reasonable intervals and with each status check you can tell the condition of each batch.
- After all the batches have completed or failed, simply retrieve every batch’s result.
- Use the initial data to compare with the results in order to determine the failed and succeeded records, then take the necessary action.
At this point during the process, as the user, you have the option to terminate the job. Terminating the job can result to prevention of batches that haven’t undergone processing from getting processed. However, it doesn’t undo the results of the batches that have already been processed.
Why do you need Salesforce Bulk API?
As a business, you will require Bulk API for successfully handling of large amounts of data. Many businesses who are experiencing daily growth find themselves overwhelmed when it comes to managing the large amounts of data that comes into their hands. They find themselves failing at their record keeping or straining too much to keep their data organized. This is why a system like Bulk API enables them to manage their data successfully.Businesses no longer have to worry about the growing number of data even when it ranges in the millions, because they have a trusted system to ensure that all their data is managed.
Bulk API limits
Bulk API has limits too and the following are some of them.Usage Limits
Using Bulk API is comes with standard usage limits. Every HTTP requests counts as a single call for the purpose of calculating the usage limits.Batch Content
Every batch has to contain a single XML or CSV file with records of a single object. If not the batch is never processed and the stateMessage is then updated. Ensure that you use enterprise WSDL in order to get the right format for the object records.Batch Limit
It is actually possible to submit up to about 5,000 batches for every 24hours and create other new batches that are associated with any job that is past the 24hour period.Batch Lifespan
Batches and jobs that have lasted more than seven days are usually eliminated from the queue despite their job status.Batch Size
Data loads that make batches are usually made up of single XMLor CSV files that are no larger than 10MB. A batch can contain no more than 10,000 records and no more than 10,000,000 characters in all the data that makes up the batch.A field can only have no more than 32,000 characters and a record has to have no more than 5,000 field and no more than 400,000 characters in all its fields.
The batch must also have some content or an error will occur.
Batch Processing Time
There is usually a five-minute limit when 100 records are processed. In case a batch takes more than 10 minutes to process, the Bulk API will place the remaining batch back in the waiting queue for it to be processed later. However, in case the batch exceed the limit which is 10 minutes after several attempts, it is again placed back into the queue and the process is repeated 10 times, after which it will be permanently labeled as failed. Even when a batch fails, some records are usually successfully completed.With your guide to Salesforce Bulk API outlined above, using it in your business should now be easier. For additional help check out how to use salesforce.