Salesforce使用Batch Class

【场景描述】：UAT过后，我们需要将客户的历史数据导进生产环境，由于记录体量很大，通常会先关闭Rules及Trigger来保证数据能快速导入，事后为了保证业务数据的合理性，我们会使用Batch来更新那些被禁用后的逻辑。

【基础知识】：
1. If you have a lot of records to process, for example, data cleansing or archiving, Batch Apex is probably your best solution.
2. This functionality has two awesome advantages:
    - Every transaction starts with a new set of governor limits
    - If one batch fails to process successfully, all other successful batch transactions aren’t rolled back.
3. You can query up to 50 million records. 
4. The default batch size is 200 records. Batches of records are not guaranteed to execute in the order they are received from the start method.
5. Batch Apex is typically stateless. Each execution of a batch Apex job is considered a discrete transaction.
6. If you specify Database.Stateful in the class definition, you can maintain state across all transactions. When using Database.Stateful, only instance member variables retain their values between transactions.

【概念图解】：Episode 7 – Asynchronous Processing in Apex

异步处理决策流程：

思考：如何利用上述流程图解决"Future method cannot be called from a future or batch method."

业务背景：每10min callout external service，然后更新Partner User的IsActive，同时需要同步Contact的Status[Active, Closed]。

原有逻辑：原本代码已经通过trigger来监测user信息是否更新，来决定是否异步同步信息到contact，简言之：trigger中调用future方法解决mix dml的问题。

现需实施：当Azure AD的group中member被移除或重新添加，SFDC需要以Azure AD作为single truth of source，来同步user的状态信息Inactive或Active。

在不清楚原逻辑会对当前实施的影响时，很容易给以下方案：
schedule + [@future(callout=true) / batch中实现callout]，然后update user

然而结合背景，我们知道这个方案的路径为：
schedule + [@future(callout=true) / batch callout] -> update user -> execute user trigger -> @future updateContact；

正好中招......

解法是啥呢？

问题分析：我们现在能确定schedule必须要，然后update user时候还不能关trigger，因为我们需要走异步同步user信息到contact。这样以来，我们唯一能动手脚的地方在schedule和callout这条路径，既然不能用Futrue Method / Batch Apex，那我们可以用Queueable来做callout就可以了呀，然后在schedule中调queueable就成功了。

问题升级：假如我们需要基于ad group的member定时批量管理partner user (add user, inactive user, active user)，同时我们需要获取member的avatar并将base64上传到中间件，从而获取图片的相对地址信息，最终更新到contact上？

问题分析：这里我们明显能感觉到callout比较多，通过email key查找是否有现存contact可以用，需要对contact, partner user和apex log做mix dml。

解决方案：chain job in queueable，先允许contact先做dml，再新开一个job将parent job的context信息传给下一个job允许user做dml，由于dml操作设为部分成功，因此再chain一个job允许apex log做dml造作，关键代码如下：

// chain a child job to seperate the dml operation -> user dml in child job
ID jobID = System.enqueueJob(new IMS_UserProvision(aduMap, email2conIdMap, rauList, sfuMap, org_alais));
System.debug(LoggingLevel.INFO, '*** jobID: ' + jobID);

归纳总结：解决"MIXED_DML_OPERATION, DML operation on setup object is not permitted"问题，大家都知道用异步，如果入参是primitive的，@future为首选；如果是non-primitive的，可以用Queueable来解决。

Queueable重点知识：

1. Each queued job runs when system resources become available. A benefit of using the Queueable interface methods is that some governor limits are higher than for synchronous Apex, such as heap size limits.
2. Using non-primitive types: Your queueable class can contain member variables of non-primitive data types, such as sObjects or custom Apex types. Those objects can be accessed when the job executes.
3. Chaining jobs: You can chain one job to another job by starting a second job from a running job. Chaining jobs is useful if your process depends on another process to have run first.
4. You can add only one job from an executing job, which means that only one child job can exist for each parent job.

Note:
1. If an Apex transaction rolls back, any queueable jobs queued for execution by the transaction are not processed.
2. Apex allows HTTP and web service callouts from queueable jobs, if they implement the Database.AllowsCallouts marker interface. In queueable jobs that implement this interface, callouts are also allowed in chained queueable jobs.
3. You can’t chain queueable jobs in an Apex test. Doing so results in an error. To avoid getting an error, you can check if Apex is running in test context by calling Test.isRunningTest() before chaining jobs.
4. You can add up to 50 jobs to the queue with System.enqueueJob in a single transaction. 
5. When chaining jobs with System.enqueueJob, you can add only one job from an executing job.

理解batch apex：

理解Scheduled Apex：

The System.Schedule method uses the user's timezone for the basis of all schedules.

【技术选型矩阵】：

【示例代码】：
Batch class

global class ExampleBatchClass implements Database.Batchable<sObject>{
    global ExampleBatchClass(){
        // Batch Constructor
    }
    // Start Method
    global Database.QueryLocator start(Database.BatchableContext BC){
        return Database.getQueryLocator(query);
    }
    // Execute Logic
    global void execute(Database.BatchableContext BC, List<sObject>scope){
        // Logic to be Executed batch wise      
    }
    global void finish(Database.BatchableContext BC){
        // Logic to be Executed at finish
    }
}

Call the batch class

ExampleBatchClass b = new ExampleBatchClass(); 
//Parameters of ExecuteBatch(context,BatchSize)
database.executebatch(b,10);

Note:
1. if batch size is not mentioned it is 200 by default.
2. 我们知道future方法不能在batch或者future方法中被调用，但是如果在trigger中调用future是可以的，前提是该trigger不能由batch触发。简言之，如果batch触发了trigger，而trigger调用了future不被允许，错误如下：
caused by: System.AsyncException: Future method cannot be called from a future or batch method

【实例1】：将Org中所有Lead历史数据的Status更新为Closed

global class LeadProcessor implements Database.Batchable<sObject>, Database.Stateful {
	// instance member to retain state across transactions
    global Integer recordsProcessed = 0;
    global Database.QueryLocator start(Database.BatchableContext bc) {
        // collect the batches of records or objects to be passed to execute
        String str = 'SELECT Id, Status FROM Lead';
        return Database.getQueryLocator(str);
    }

    global void execute(Database.BatchableContext bc, List<Lead> records){
        // process each batch of records
        List<Lead> ldList = new List<Lead>();
        for(Lead l : records) {
            l.Status = 'Closed';
            ldList.add(l);
            recordsProcessed ++;
        }
        update ldList;
    }    

    global void finish(Database.BatchableContext bc){
        // execute any post-processing operations
        System.debug(recordsProcessed + ' records processed.');
        AsyncApexJob job = [SELECT Id, Status, NumberOfErrors, 
                            JobItemsProcessed,
                            TotalJobItems, CreatedBy.Email
                            FROM AsyncApexJob
                            WHERE Id = :bc.getJobId()];
    }    

}

finish方法在batch执行完毕执行，一般用作邮件通知，代码如下：

global void finish(Database.BatchableContext BC){
   // Get the ID of the AsyncApexJob representing this batch job
   // from Database.BatchableContext.
   // Query the AsyncApexJob object to retrieve the current job's information.
   AsyncApexJob a = [SELECT Id, Status, NumberOfErrors, JobItemsProcessed,
      TotalJobItems, CreatedBy.Email
      FROM AsyncApexJob WHERE Id =
      :BC.getJobId()];
   // Send an email to the Apex job's submitter notifying of job completion.
   Messaging.SingleEmailMessage mail = new Messaging.SingleEmailMessage();
   String[] toAddresses = new String[] {a.CreatedBy.Email};
   mail.setToAddresses(toAddresses);
   mail.setSubject('Apex Sharing Recalculation ' + a.Status);
   mail.setPlainTextBody
   ('The batch Apex job processed ' + a.TotalJobItems +
   ' batches with '+ a.NumberOfErrors + ' failures.');
   Messaging.sendEmail(new Messaging.SingleEmailMessage[] { mail });
}

单元测试代码：

@isTest
private class LeadProcessorTest {
    @testSetup 
    static void setup() {
        List<Lead> ldList = new List<Lead>();
        for(Integer i = 0; i < 200; i++) {
            ldList.add(new Lead(LastName = 'Lead' + i, Company = 'com' + i, Status = 'Open'));
        }
        insert ldList;
    }
    static testMethod void test1() {
        Test.startTest();
        LeadProcessor lp = new LeadProcessor();
        Id batchId = Database.executeBatch(lp);
        Test.stopTest();
    }
}

在Developer Console的Open Excute Anonymous Window执行以下代码：

LeadProcessor leadBach = new LeadProcessor(); 
Id batchId = Database.executeBatch(leadBach, 100);

【参考资料】：
https://trailhead.salesforce.com/modules/asynchronous_apex/units/async_apex_batch
补充：
Batch特性：如果我们在start里面写查询语句，如果查询中包含关系字段（除关系ID外），那么debug时是出不来关系字段的值的，这时候如果用公式字段代替关系字段，同样debug不出值；如果我们在excute里面做查询，那么debug出来的只有ID和Name的值，其他字段值也出不来。

举例：我们需要处理Account和Contact，在start里面查询语句为：select id, name, accountid, account.site from contact,那么只能debug出id, name, accountid的值；如果我们在excute里面写入：select id, name, site from account，只能debug出id和name的值。
建议：如果是跨对象的过滤，首先将子对象记录在start中查处来（Id），然后在excute里面将查出来的id作为参数传入外部异步方法，该方法主要用来使用select再次查询，这个时候就可以debug出关系字段了，意思是就可以做过滤了。
参考文档：http://www.iterativelogic.com/developing-custom-apex-rollup-summary-field-logic/

【纠偏与建议篇180802】：
早期偏见：batch查询跨对象（父查子，子查父），那么由于debug不出跨对象数据，那么自然无法使用此类值做逻辑判断。
下面代码两种组织思维（见注释/非注释部分），同样实现了将所有子记录的Name值以；隔开拼起来填到父记录某字段的需求。

global class BatchAccountLoginsPopulate implements Database.Batchable<sObject>, Database.Stateful {

    global Integer recordsProcessed = 0;

    global Database.QueryLocator start(Database.BatchableContext bc) {
        // String query = 'SELECT Id, Account__c FROM MT4_Accounts__c';
        String query = 'SELECT Id, Name, Trading_Account_Login__c, (SELECT Id FROM MT4_Accounts__r) FROM Account WHERE Trading_Account_Login__c  = NULL';
        return Database.getQueryLocator(query);
    }

    // global void execute(Database.BatchableContext BC, list<MT4_Accounts__c> records) {
    global void execute(Database.BatchableContext BC, list<Account> records) {
        // List<Account> accList = new List<Account>();
        // Set<String> accIdSet = new Set<String>();
        // for(MT4_Accounts__c rAcc : records) {
        //     accIdSet.add(rAcc.Account__c);
        // }
        // if(accIdSet.size() > 0) {
        //     TradingAccountTriggerFunction.joinLoginToAccount(accIdSet);
        // }
        Set<String> accIds = new Set<String>();
        for(Account acc : records) {
            if(acc.MT4_Accounts__r.size() > 0) {
                accIds.add(acc.Id);
            }
        }
        if(accIds.size() > 0) {
            TradingAccountTriggerFunction.joinLoginToAccount(accIds);
        }
    }

    global void finish(Database.BatchableContext BC) {
        AsyncApexJob a = [SELECT Id, Status, NumberOfErrors, JobItemsProcessed,
                          TotalJobItems, CreatedBy.Email
                          FROM AsyncApexJob WHERE Id = :BC.getJobId()];
        System.debug('[Apex Sharing Recalculation ' + a.Status + ']: The batch Apex job processed ' + a.TotalJobItems +
        ' batches with '+ a.NumberOfErrors + ' failures.');
    }
}

写测试类可能遇到如下错误：原因是该字段字段类型为长文本字段

System.QueryException: field 'Trading_Account_Login__c' can not be filtered in a query call

【实践总结180802】：
写batch最佳实践是query部分最好以参数形式传进来，hard code会导致不灵活，如果部署到了生产环境刷数据如果有最新查询方案，动态传参会省大笔修改逻辑，写测试类，重新部署，修正错误等时间。

【Execute All Batch Classes180814】：

One Universal Schedule Class to Execute All Batch Classes In Salesforce

Are you creating schedule class for each batch class if you need to schedule that?

If yes, stop doing like that and start creating a universal (common scheduler class) like below and use it instead of creating multiple schedules.

Step 1: 
Create a universal scheduler class

global class UniversalScheduler implements Schedulable {
	global Database.Batchable<SObject> batchClass{get;set;}
    global Integer batchSize{get;set;} {batchSize = 200;}

    global void execute(SchedulableContext sc) {
    	Database.executebatch(batchClass, batchSize);
    }
}

Step 2:

Let say, You have two batch classes and you want to schedule, then schedule batch class like below using UniversalScheduler class

Batch Class 1:

AccountBatchProcess accBatch = new AccountBatchProcess(); // Batch Class Name
UniversalScheduler scheduler = new UniversalScheduler();
scheduler.batchClass = accBatch;
scheduler.batchSize = 100;
String sch = '0 45 0/1 1/1 * ? *';
System.schedule('Account Batch Process Scheduler', sch, scheduler);

Batch Class 2:

ContactBatchProcess cntBatch = new ContactBatchProcess(); // Batch Class Name
UniversalScheduler scheduler = new UniversalScheduler();
scheduler.batchClass = cntBatch;
scheduler.batchSize = 500;
String sch = '0 45 0/1 1/1 * ? *';
System.schedule('Contact Batch Process Scheduler', sch, scheduler);

【每隔5min执行一次Schedule方法】：

System.schedule('IMS_ScheduleCampMemberStatusUpdate 1',  '0 00 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 2',  '0 05 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 3',  '0 10 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 4',  '0 15 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 5',  '0 20 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 6',  '0 25 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 7',  '0 30 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 8',  '0 35 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 9',  '0 40 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 10', '0 45 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 11', '0 50 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());
    System.schedule('IMS_ScheduleCampMemberStatusUpdate 12', '0 55 * * * ?', new IMS_ScheduleCampMemberStatusUpdate());

【Apex批量删除Schedule方法】：

List<CronTrigger> JOBIDLIST = new List<CronTrigger>();
JOBIDLIST = [SELECT Id, CronJobDetail.name, NextFireTime, PreviousFireTime, State, StartTime, EndTime, CronExpression FROM CronTrigger where CronJobDetail.name like 'IMS_Schedule%'];
for(CronTrigger job:JOBIDLIST )
    System.abortJob(job.id);

【其他话题】：
1. Using Aggregate SOQL queries/results in Batch Apex
2. Executing Batch Apex in Sequence
3. Cron Expression Generator & Explainer - Quartz