Send bulk data from Salesforce to external API

Author - Webner
5.02.2018
|
0 Comments
|

Scenario : We have the situation where we want to create a backup of all the contacts in Salesforce on daily basis to AWS S3. We are using PHP code to create files according to the desired criteria and upload them to AWS S3. A crucial step is to send data to our PHP code as we want all the records which are created or updated today. Also, we want data from all the fields in contact object (approximately 120 fields).

Initially, we created a scheduler to make it run daily to fetch all the contacts which are created and updated in last 24 hours using the asynchronous method i.e future method and will send data in JSON format to PHP code. As the data in our case was very large so the process was failing due to the maximum heap limit and was unable to process the JSON. To overcome this, we tried to create data chunks and called a future method in a loop, but we got “Total number of callouts” limits exceed exception.

Solution : Finally we resolved this issue by using this solution :

Call the HTTP method from batch class.

Note : The future method cannot be called from batch class. So we have to make the function as non-future method :

In “start” method of Batch Class, we queried the records which we need and created chunks of records so that the JSON should not exceed heap size limit.

From “Execute” method we created the JSON of fetched records and posted to the HTTP request (our PHP script).

Below is the sample code :

Salesforce Batch class :
global class Batch_UploadContactsInAWS implements Database.Batchable , Database.AllowsCallouts, Database.Stateful{
    
    global List contactList;
    global List start(Database.BatchableContext BC){
        List contactList = [Select Id,AccountId,LastName,FirstName,Name,OtherStreet,OtherCity,OtherState,OtherPostalCode,OtherCountry,MailingStreet,MailingCity,MailingState,MailingPostalCode,MailingCountry,Phone,Fax,MobilePhone,HomePhone,OtherPhone,Email,Title,Department,AssistantName,LeadSource,Birthdate,Description,OwnerId,CreatedDate,LastModifiedDate,PhotoUrl,Jigsaw,JigsawContactId,FieldName_1,FieldName_2,FieldName_3,FieldName_4,FieldName_5,FieldName_6,FieldName_7,FieldName_8,FieldName_9,FieldName_10,FieldName_11,FieldName_12,FieldName_13,FieldName_14,FieldName_15,FieldName_16,FieldName_17,FieldName_18,FieldName_19,FieldName_20,FieldName_21,FieldName_22,FieldName_23,FieldName_24,FieldName_25,FieldName_26,FieldName_27,FieldName_28,FieldName_29,FieldName_30,FieldName_31,FieldName_32,FieldName_33,FieldName_34,FieldName_35,FieldName_36,FieldName_37,FieldName_38,FieldName_39,FieldName_40,FieldName_41,FieldName_42,FieldName_43,FieldName_44,FieldName_45,FieldName_46,FieldName_47,FieldName_48,FieldName_49,FieldName_50,FieldName_51,FieldName_52,FieldName_53,FieldName_54,FieldName_55,FieldName_56,FieldName_57,FieldName_58,FieldName_59,FieldName_60,FieldName_61,FieldName_62,FieldName_63,FieldName_64,FieldName_65,FieldName_66,FieldName_67,FieldName_68,FieldName_69,FieldName_70,FieldName_71,FieldName_72,FieldName_73,FieldName_74,FieldName_75,FieldName_76,FieldName_77,FieldName_78,FieldName_79,FieldName_80,FieldName_81,FieldName_82,FieldName_83,FieldName_84,FieldName_85,FieldName_86,FieldName_87,FieldName_88,FieldName_89,FieldName_90,FieldName_91,FieldName_92,FieldName_93,FieldName_94,FieldName_95,FieldName_96,FieldName_97,FieldName_98,FieldName_99,FieldName_100 from Contact where createdDate=YESTERDAY];
        return contactList;
    }

    global void execute(Database.BatchableContext BC, List contacts){
        String jsonstr = (String)JSON.serialize(contacts );
        UploadContactsInAWS.updatetRecordToS3(jsonstr);
    }

    global void finish(Database.BatchableContext BC){
    }
}

HTTP class :

Calling Batch class :

public class UploadContactsInAWS{

 public void saveContactsToS3(){
        Batch_UploadContactsInAWS p= new Batch_UploadContactsInAWS();
        Database.executeBatch(p,2000);// Calling batch class with 2000 chunks
    }

//HTTP callout method
    public static void updatetRecordsToS3( String jsonStr)
    {
        try
        {
            Http h = new Http();
            HttpRequest req = new HttpRequest();
            HttpResponse res = new HttpResponse();
            String url = 'http://XX.XXX.XX/********************/test.php';
            req.setEndPoint(url);
            req.setCompressed(true); // Compress the JSON
            req.setTimeout(120000);  //Increase callout time
            req.setMethod('POST');
            //req.setHeader('Content-Type' ,'application/json');
            String encodedStr=EncodingUtil.urlEncode(jsonStr , 'UTF-8'); // URL encoding to  avoid special characters
            req.setBody(encodedStr); 
            if(!Test.isRunningTest()){  
                res = h.send(req);
                string response  = res.getBody();
            }
        }
        catch(exception e)
        {
            system.debug('Save file to server exception :'+e);
        }
    } 
    
}
Webner Solutions is a Software Development company focused on developing Insurance Agency Management Systems, Learning Management Systems and Salesforce apps. Contact us at dev@webners.com for your Insurance, eLearning and Salesforce applications.

Leave a Reply

Your email address will not be published. Required fields are marked *

Send your message