Batch processing allows you to process multiple objects in an Object Storage Service (OSS) bucket at a time. For example, you can convert the format of objects whose names contain the .docx suffix in a directory of a bucket to PDF and transcode the format of the objects from MOV to MP4.
You can call the new version of API operations and use the new version of SDKs in the regions supported by Intelligent Media Management (IMM).
Usage notes
A batch processing job creates data processing tasks for all objects that meet specific conditions. We recommend that you run a batch processing job for a small number of objects for testing.
If a batching processing job is suspended, no new data processing tasks are generated, but submitted data processing tasks are not affected.
To avoid unexpected behaviors such as loops caused by a trigger and source object overwrites, do not specify an output path that is prefixed with the input path.
The output path cannot end with a forward slash (
/). The output path must contain the object name with an extension. Variables can be used in the object name. For more information about variables, see Variables.If the output path does not end with a forward slash (
/), include at least one variable in the name of the output object. Otherwise, multiple output objects may overwrite each other.If you use the {key}, {barename}, or {basename} variable as the output path, make sure that the output path is different from the input path. Otherwise, source objects are overwritten or the tasks run in a loop.
Create a batch processing job
Log on to the OSS console.
In the left-side navigation pane, click Buckets. On the Buckets page, click the bucket for which you want to create a batch processing job.
In the left-side navigation pane, click Data Processing.
In the left-side navigation pane, click Document Processing, Media Processing, or other data processing capabilities provided by Intelligent Media Management (IMM) based on your business requirements. In the following examples, Document Processing is clicked.
Click the Batch Processing tab, and then click Create Batch Processing.
In the Create Batch Processing panel, configure parameters in the Input Configuration, Output Configuration, Permission Configuration, and Notification Configuration sections.
Parameter
Description
Input Storage Bucket
The name of the bucket in which the objects that you want to process are stored.
Mode
Valid values: Select File or Select Prefix.
Select File: Data processing is performed on multiple objects that you select.
Select Prefix: Data processing is performed on objects that meet specific rules in a specific directory.
Input Path
If you select Select File for Mode, you can select one or more objects for data processing.
Path Prefix
If you select Select Prefix for Mode, you can specify a prefix for the path of the objects and use a filter rule to select the objects for data processing.
Filter Rule
If you select Select Prefix for Mode, you can specify a path prefix and filter rule to filter the objects that you want to process.
All: All objects in the directory specified by the Path Prefix parameter are processed.
Prefix: Objects whose names contain a specific prefix in the directory specified by the Path Prefix parameter are processed. Separate multiple prefixes with commas (,).
Suffix: Objects whose names contain a specific suffix in the directory specified by the Path Suffix parameter are processed. Separate multiple suffixes with commas (,).
Style
The processing style for the objects. If no processing style is available, select Add Style from the drop-down list to create a processing style.
Output Path
The storage path of the processed objects.
The storage path cannot end with a forward slash (/). We recommend that you use variables in the output path. Otherwise, multiple output objects may overwrite each other. For more information, see Variables.
Service Role
The service role. IMM assumes the service role to access resources in other cloud services, such as OSS. We recommend that you select
AliyunIMMBatchTriggerRole. You can create the AliyunIMMBatchTriggerRole role on the Cloud Resource Access Authorization page.Message Queue
The Message Service (MNS) topic. When a data processing job is complete, processing results are sent to MNS by using a notification. For more information, see Overview.
Click OK.
After a batch processing job is created, the corresponding data processing tasks are generated for each existing object. When the tasks run, corresponding fees are generated. For more information, see Billable items.
Query a batch processing job
Log on to the OSS console.
In the left-side navigation pane, click Buckets. On the Buckets page, click the bucket for which you want to query a batch processing job.
In the left-side navigation pane, click Data Processing.
In the left-side navigation pane, click Document Processing, Media Processing, or other data processing capabilities provided by IMM based on the type of the objects.
Click the Batch Processing tab to view the data processing tasks of the batch processing job. Click the page up and down icon in the lower part of the page to view the previous or next page.
The following table describes the status of the data processing tasks of the batch processing job.
Status
Description
Ready
The data processing task is newly created and ready.
Running
The data processing task is running.
Succeeded
The data processing task is complete.
Suspended
The data processing task is suspended.
Failed
The data processing task failed and cannot be automatically recovered.
Enter an ID in the Batch Processing ID search box and click the search icon to search for a specific batch processing job.
Click Details in the Actions column to view the details of the batch processing job.
Click the icon next to a batch processing ID to view information about all data processing tasks of a batch processing job. You can search for a data processing task based on the source file path and status. You can click Details in the Actions column to view the details of the data processing task.