Mongo DB Data To Salesforce

Move Data from MongoDB to Salesforce

Posted 06/24/2016 by Kapil Kamtikar

In one of my previous project there was requirement to create a utility that will move data from MongoDB to Salesforce on regular interval. I am sharing what I have implemented to achieve this.

One of the simple ways to migrate data from MongoDB to Salesforce is using MongoDB API and Salesforce API.

Following are the steps involved in the process,

  1. Store data in MongoDB
  2. Create Apex web service for storing data into Salesforce
  3. Get the unprocessed records from MongoDB
  4. Pass unprocessed records to Salesforce web service which will store the data into Salesforce
  5. Automate the process

1 Storing Data to MongoDB

 Let us consider a simple example, we have a form where we are requesting user to enter his email id and on submit of this form we are storing email id into MongoDB  Following code is example for adding email id into MongoDB. Here we need to provide connection string of MongoDB. By using MongoDB API we can add data. 

public void IsertUserDetailsToMongoDB (string Email,string IP)
        {
     string connectionString = "mongodb://localhost:27017";
            var client = new MongoClient(connectionString);
            var server = client.GetServer();

            //Use the MongoClient to access the server
            var database = server.GetDatabase(ConfigurationSettings.AppSettings["MongodbUserDB"]);

            //get mongodb collection
            var collection = database.GetCollection<bsondocument> ("Users");
             BsonDocument userDetals = new BsonDocument { { "Email", Email }, { "IP", IP },{ "IsSync", 0 } };

            collection.Insert(userDetals);            

        }

Also we need to keep track which data got sync with Salesforce, so we’ll add a flag “IsSync” and set the default value to 0. Once the sync process is done we’ll set this flag as 1. So whenever the MongoDB to Salesforce sync process will start that time application will pick the records which are not sync or records with 0 values.

2 Create Apex web service in Salesforce

To store the data into Salesforce we need to create a custom object and custom object fields. Custom object fields store the data for your custom object record.

Please refer this link for creating custom object in Salesforce.

Now Apex web service class needs to be created: Apex class methods can be exposed as custom SOAP Web service calls. This allows an external application to invoke an Apex Web service to perform an action in Salesforce. Using this Apex web service we will store details from external system.

Following is the code for Apex service class in this UserInfo__c is the custom Object and UserEmail__c and IP__c are the fields of custom object. 

global class AddUserDetailsWebService
        {
            WebService static string AddUserDetails(String Email, String IP)
            {
                string response;
                UserInfo__c sub = new UserInfo__c()
                sub.UserEmail__c = Email;
                sub.IP__c = IP;
                try
                {
                    insert sub;
                    response = 'success';
                }
                catch (DmlException ex)
                {
                    response = ex.getMessage();
                }
                return response;
            }
        }

External application will invoke this web service method and store User information into Salesforce.

3. Get the unprocessed records from MongoDB

From MongoDB we need to get the records which are not moved into Salesforce. We can get these records with the help of IsSync column. We’ll fetch all the records (List of userdetails) with IsSync value 0 (i.e. unprocessed records) Following is the code for the same.

public class UserDetailsModel
{
    public string Email { get; set; }
    public string IP { get; set; }
    public int IsSync { get; set; }
}
public class MongoDBUsers
    {
public List<userdetailsmodel> GetUserDetailList()
{
   var client = new MongoClient(connectionString);
   var server = client.GetServer();

            //Use the MongoClient to access the server MongodbUserDB
   var database = server.GetDatabase(ConfigurationSettings.AppSettings["MongodbUserDB"]);
            //get mongodb collection
            var collection = database.GetCollection<bsondocument> ("Users");
            var mongoDBlist = collection.FindAll().ToList();
            
            List<userdetailsmodel> userDetailList = new List<userdetailsmodel>();

            foreach (var item in mongoDBlist)
            {
                
                UserDetailsModel userItem = new UserDetailsModel();
                foreach (var childItem in item.Elements)
                {
                    
                    if (childItem.Name.ToString() == "Email")
                    {
                        userItem.Email = childItem.Value.ToString();
                    }                    

                    if (childItem.Name.ToString() == "IP")
                    {
                        userItem.IP = childItem.Value.ToString();
                    }

                    if (childItem.Name.ToString() == "IsSync")
                    {
                        userItem.IsSync = (int)childItem.Value;
                    }

                }                                               
                userDetailList.Add(userItem);
            }
            return userDetailList.Where(a => a.IsSync == 0).ToList();
            
     }
}

4. Pass unprocessed records to Salesforce web service which will store the data into Salesforce

Here we have list of unprocessed data, now we need to pass this data to Apex web service class which will create record into Salesforce.

To access the apex class from Salesforce we need to add web service reference to the project. There is a blog in which adding service reference of an apex class in .Net application is explained well . Please add the web service reference and do the settings as mentioned in this link

After adding web service reference and providing proper authentication we can use Apex web service class and pass parameters, following is the code for the same.

public void addUserDetailsToSalesforce()
        {
            //Salesforce Apex service class 
            AddUserDetailsWebService userDetalsBinding = new AddUserDetailsWebService();
            
            //MongoDb user list which are not sync with Salesforce 
            MongoDBUsers userDetails = new MongoDBUsers();
            var unsyncUserList = userDetails.GetUserDetailList();

            if (unsyncUserList.Any())
            {
                // Check if Salesforce web service already connected then no need to call partner service for get session id  
                if (!IsConnected())
                    getSessionInfo();
                //Validate the web service session 
                userDetalsBinding.SessionHeaderValue = new AddUser.SessionHeader();
                userDetalsBinding.SessionHeaderValue.sessionId = lr.sessionId;
                
                foreach (var unsyncUserItem in unsyncUserList)
                {
                    //Salesforce Apex webservice method 
                   userDetalsBinding.AddUserDetails(unsyncUserItem.Email, unsyncUserItem.IP);
                   userDetails.UpdateUser(unsyncUserItem);
                  }                
            }        
        } 

Once the data created into Salesforce we need to set IsSync flag to 1 in MongoDB, this will mark all the records as 1 which moved from MongoDB to Salesforce, So that when next cycle of Data movement these records will not be processed. Following is the code for updating record in MongoDB.

public void UpdateUser(UserDetailsModel user)
        {
            var client = new MongoClient(connectionString);
            var server = client.GetServer();

            //Use the MongoClient to access the server
            var database = server.GetDatabase(ConfigurationSettings.AppSettings["MongodbUserDB"]);

            //get mongodb collection
            var collection = database.GetCollection <bsondocument>("Users");
           
            var mongoDBlist = collection.FindAll();
	    //get the current record with email id 
            var query = Query.EQ("Email", user.Email);

            var result = collection.FindOne(query);
            var update = Update.Set("IsSync", 1); 
            collection.Update(query, update);
        }


5. Automate Process

We can automate this process in different ways as given below

  1. We can create a scheduler in the Sitecore and call the Data movement (like storing date to MongoDB and Sending /storing data into Salesforce) code by scheduler. We can run this process in specific interval.  
  2. We can create exe for data movement (like storing date to MongoDB and Sending /storing data into Salesforce) code and call this exe by creating a windows scheduler at specific interval. 

And this is how in few steps we can move data from MongoDB to Salesforce. Hope this blog will help some one how is looking for moving data from MongoDB to Salesforce.

Share:

Add your comment

 
 

 

Comments

  • arjunan said:
    7/8/2016 8:27 PM

    Thanks for sharing an excellent article. It could have been better if you shared the code in Github :)

  • Patrick Wensel said:
    12/20/2016 12:15 AM

    Thank you for sharing your experiences and practical knowledge. This information will definitely help developers if they will be assigned with the same task. Keep up the good work!!

Archive

Syndication