To ensure that your data synchronization tasks and data scheduling tasks in DataWorks run as expected, you must establish a network connection between the virtual private cloud (VPC) with which your resource group is associated and the data source that you want to access. The data source can be a database, a data service, or other data in a network environment. This topic describes the network connectivity solutions for data sources that are deployed in different types of network environments.
Background information
Most features of DataWorks (such as creating data sources, data synchronization, DataAnalysis, data collection, and DataService Studio) require operations based on connected data sources or computing resources. If the data source you need to access is not in the VPC to which the DataWorks resource group is bound (for example, the data source is in another VPC or IDC), you need to choose an appropriate network connectivity solution based on different network situations to establish connectivity between the VPC bound to the DataWorks resource group and the network where the data source is located.
For example, when you configure a data synchronization task, you must establish a network connection between the VPC with which your resource group is associated and the source and a network connection between the VPC and destination.
Prerequisites
You have purchased a resource group of an appropriate specification. For more information about how to purchase a resource group, see Create and use a serverless resource group.
For more information about resource groups, see Overview of DataWorks resource groups.
The network connectivity solutions provided in this topic apply only to serverless resource groups and the exclusive resource group for Data Integration, exclusive resource group for scheduling, and exclusive resource group for DataService Studio in old-version resource groups.
Notes
You can associate a serverless resource group with a virtual private cloud (VPC) to enable the resource group to access a data source or an address in a complex network environment over an internal network. However, serverless resource groups do not have Internet access capabilities by default. If you need to access a data source or network over the Internet, you must configure an Internet NAT gateway and an EIP for the VPC with which the serverless resource group is associated. For more information, see Solution 6: Connect to a data source over the Internet.
The speed and stability of tasks over the Internet cannot be guaranteed. We recommend that you synchronize data over an internal network or Cloud Enterprise Network.
Network connectivity is an important factor that affects the running result of your task.
Network connections cannot be established between a resource group and data sources that are deployed in the classic network. If the data source or business that you want to access is deployed in the classic network, we recommend that you migrate the data source or business to a VPC.
Network connectivity solutions
The choice of network connectivity solution depends on the relationship between the data source and the resource group of the DataWorks workspace. Choose the appropriate network connectivity solution based on your actual situation:
Solution 1: Connect to a data source that uses the same Alibaba Cloud account and is deployed in the same region
Scenarios
We recommend that you use this solution if your data source and DataWorks workspace meet all the following conditions:
The data source is an Alibaba Cloud service.
The data source and the DataWorks workspace use the same Alibaba Cloud account.
The data source and the DataWorks workspace are deployed in the same region.
Solution description
In scenarios where the same account and region are used, we recommend that you use a VPC network (internal network) connection. Deploy the resource group of the DataWorks workspace and the data source in the same VPC to establish network connectivity.
Network connectivity diagram
Configure network connectivity
For detailed solution description and steps, see Solution 1: Connect to a data source that uses the same Alibaba Cloud account and is deployed in the same region.
Solution 2: Connect to a data source that uses the same Alibaba Cloud account but is deployed in a different region
Scenarios
We recommend that you use this solution if your data source and DataWorks workspace meet all the following conditions:
The data source is an Alibaba Cloud service.
The data source and the DataWorks workspace use the same Alibaba Cloud account.
The data source and the DataWorks workspace are deployed in different regions.
Solution description
In scenarios where the same account but different regions are used, we recommend that you use a VPC network (internal network) connection. Use network connectivity tools (Cloud Enterprise Network or VPC peering connection) to connect the VPC of the resource group of the DataWorks workspace with the VPC where the data source is located to establish network connectivity.
Network connectivity diagram
Configure network connectivity
For detailed solution description and steps, see Solution 2: Connect to a data source that uses the same Alibaba Cloud account but is deployed in a different region.
Solution 3: Connect to a data source that uses a different Alibaba Cloud account
Scenarios
We recommend that you use this solution if your data source and DataWorks workspace meet all the following conditions:
The data source is an Alibaba Cloud service.
The data source and the DataWorks workspace use different Alibaba Cloud accounts.
Solution description
In scenarios where different Alibaba Cloud accounts are used, we recommend that you use a VPC network (internal network) connection. Use network connectivity tools (Cloud Enterprise Network or VPC peering connection) to connect the data source of Account A with the resource group of the DataWorks workspace of Account B to establish network connectivity.
Network connectivity diagram
Configure network connectivity
For detailed solution description and steps, see Solution 3: Connect to a data source that uses a different Alibaba Cloud account.
Solution 4: Connect to a data source that is deployed on an ECS instance
Scenarios
We recommend that you use this solution if your data source meets the following condition:
The data source is deployed on an Alibaba Cloud ECS instance.
Solution description
If the ECS instance where the data source is deployed and the DataWorks workspace use the same Alibaba Cloud account and are deployed in the same region, we recommend that you use a VPC network (internal network) connection. Deploy the resource group of the DataWorks workspace and the ECS instance in the same VPC to establish network connectivity.
If the ECS instance where the data source is deployed and the DataWorks workspace use different Alibaba Cloud accounts or use the same Alibaba Cloud account but are deployed in different regions, we recommend that you use a VPC network (internal network) connection. Use network connectivity tools (Cloud Enterprise Network or VPC peering connection) to connect the VPC of the resource group of the DataWorks workspace with the VPC where the ECS instance is located to establish network connectivity.
Network connectivity diagram
Same Alibaba Cloud account and same region
Same Alibaba Cloud account but different regions
Different Alibaba Cloud accounts
Configure network connectivity
For detailed solution description and steps, see Solution 4: Connect to a data source that is deployed on an ECS instance.
Solution 5: Connect to a data source that is deployed in an on-premises data center
Scenarios
We recommend that you use this solution if your data source meets the following condition:
The data source is deployed in an on-premises data center.
Solution description
If the data source is deployed in an on-premises data center, we recommend that you use a VPC network (internal network) connection. Use a network connectivity tool (Express Connect) to connect the on-premises network where the data source is located with the VPC of the resource group of the DataWorks workspace to establish network connectivity.
Network connectivity diagram
Configure network connectivity
For detailed solution description and steps, see Solution 5: Connect to a data source that is deployed in an on-premises data center.
Solution 6: Connect to a data source over the Internet
Scenarios
We recommend that you use this solution if your data source meets the following condition:
The data source has an Internet endpoint.
Solution description
Serverless resource groups do not have Internet access capabilities by default. You need to configure an Internet NAT gateway and an EIP for the VPC with which the resource group is associated before you can access data sources over the Internet.
Old-version resource groups can access the Internet and can directly connect to the data source over the Internet.
NoteOld-version resource groups are being phased out. We recommend that you use serverless resource groups.
Network connectivity diagram
The diagram applies only to serverless resource groups. Old-version resource groups are already associated with EIPs and can directly connect to data sources.
Configure network connectivity
For detailed solution description and steps, see Solution 6: Connect to a data source over the Internet.
References
For more information about resource groups, see Overview of DataWorks resource groups.
For information about how to create and use resource groups, see Create and use a serverless resource group.
For information about how to associate a resource group with a VPC, see Associate with a VPC.
Configure an SNAT entry on the Internet NAT gateway for the VPC and vSwitch with which the resource group is associated. For more information, see Access the Internet by using SNAT of an Internet NAT gateway.
For common questions about network connectivity, see Resource group operations and network connectivity.