Access HTTPS service using IP - MaxCompute - Alibaba Cloud Documentation Center

Remote VPC services support access exclusively through domain names. Direct access using IP will trigger an error, causing the request to fail. This topic describes how to solve this problem by adding a domain name to the request host for accessing HTTPS services through an IP address, fulfilling the remote access requirements for tasks such as Spark or UDF.

Direct access to HTTPS service by using IP failed

Problem description

The HTTPS feature is essential when accessing remote VPC services such as KMS and OSS for Spark or UDF tasks. However, directly accessing these services by using IP results in the error message shown below:

SSL: no alternative certificate subject name matches target host name '47.116.XX.XX'
More details here: https://curl.haxx.se/docs/sslcerts.html 
curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. 
To learn more about this situation and how to fix it, please visit the web page mentioned above.

Solution

To resolve the issue of SSL certificate verification failure when directly accessing HTTPS services by using IP, add the domain name to the request host.

1. Retrieve the IP information of the remote service.

Use ping command

Enter the following command in the Windows or Linux console to retrieve the remote service IP:

ping service.cn-shanghai-vpc.maxcompute.aliyun-inc.com

The result returned in the Windows environment is as follows:
The result returned in the Linux environment is as follows:

Use dig command

Install Bind-utils in your environment.
- Windows
  Download BIND9.17.12.x64.zip, extract it to a specified directory, such as D:\install\BIND9.17.12.x64, and add this path to the Path environment variable in Windows.
- Linux (CentOS)
  Execute the command sudo yum install bind-utils in the Linux console to install.
Execute the following command in the console:
```
dig service.cn-shanghai-vpc.maxcompute.aliyun-inc.com
```
The result returned is as follows:
Windows
Linux

2. Configure the IP information.

Refer to the following code samples to add IP information to the access_url of the request, based on the Python version you are using. Perform remote access tests in the corresponding network environment before deploying the task.

Python 2

# _*_ coding: utf-8 _*_
# only for python2
import requests
from urlparse import urlparse

class HostHeaderSSLAdapter(requests.adapters.HTTPAdapter):
    def __init__(self, resolved_ip):
        super(HostHeaderSSLAdapter,self).__init__()
        self.resolved_ip = resolved_ip

    def send(self, request, **kwargs):
        connection_pool_kwargs = self.poolmanager.connection_pool_kw
        result = urlparse(request.url)

        if result.scheme == 'https' and self.resolved_ip:
            request.url = request.url.replace(
                'https://' + result.hostname,
                'https://' + self.resolved_ip,
            )
            connection_pool_kwargs['assert_hostname'] = result.hostname
            request.headers['Host'] = result.hostname
        else:
            connection_pool_kwargs.pop('assert_hostname', None)

        return super(HostHeaderSSLAdapter, self).send(request, **kwargs)

def access_url(url, resolved_ip):
    session = requests.Session()
    # Get the domain name part from the url
    parsed_url = urlparse(url)
    hostname = parsed_url.hostname

    session.mount('https://'+hostname, HostHeaderSSLAdapter(resolved_ip))
    try:
        r = session.get(url)
    except Exception as e:
        print("err : "+ str(e))
    else:
        if r.status_code != 200:
            print("not 200 " + ",resp : "+ r.text)
        else:
            print("success" + ",resp : "+ r.text)

if __name__ == "__main__":
    # Retrieve IP through the dig command, need to dig in VPC environment
    # VPC address test
    # access_url("https://service.cn-shanghai-vpc.maxcompute.aliyun-inc.com", "100.103.104.45")
    # Public network address test
    access_url("https://service.cn-shanghai.maxcompute.aliyun.com", "47.116.XX.XX")

Python 3

# _*_ coding: utf-8 _*_
import requests
from urllib.parse import urlparse 

class HostHeaderSSLAdapter(requests.adapters.HTTPAdapter):
    def __init__(self, resolved_ip):
        super().__init__()                                        
        self.resolved_ip = resolved_ip

    def send(self, request, **kwargs):
        connection_pool_kwargs = self.poolmanager.connection_pool_kw
        result = urlparse(request.url)

        if result.scheme == 'https' and self.resolved_ip:
            request.url = request.url.replace(
                'https://' + result.hostname,
                'https://' + self.resolved_ip,
            )
            connection_pool_kwargs['server_hostname'] = result.hostname  
            # overwrite the host header
            request.headers['Host'] = result.hostname
        else:
            # these headers from a previous request may have been left
            connection_pool_kwargs.pop('server_hostname', None)           
        return super().send(request, **kwargs)

def access_url(url, resolved_ip):
    session = requests.Session()
    # Get the domain name part from the url
    parsed_url = urlparse(url)
    hostname = parsed_url.hostname

    session.mount('https://'+hostname, HostHeaderSSLAdapter(resolved_ip))
    try:
        r = session.get(url)
    except Exception as e:
        print("err : "+ str(e))
    else:
        if r.status_code != 200:
            print("not 200 " + ",resp : "+ r.text)
        else:
            print("success" + ",resp : "+ r.text)

if __name__ == "__main__":
    # Retrieve IP through the dig command, need to dig in VPC environment
    # VPC address test
    # access_url("https://service.cn-shanghai-vpc.maxcompute.aliyun-inc.com", "100.103.104.45")
    # Public network address test
    access_url("https://service.cn-shanghai.maxcompute.aliyun.com", "47.116.XX.XX")

Test results

Note

To access instance services within the VPC network, configure the Python environment in the VPC network, and replace the service link and IP address for testing.

Access the MaxCompute service locally through the public network.
Access the MaxCompute service on an Elastic Compute Service instance running Linux through the public network.