Compress Large Python Files

We have to work with large size files in Python. If we use the large files, they occupy more space, and it isn't easy to handle the large files too. In such cases, we have a module in Python to work with the large files.

The module we can use to handle large files is zipfile. By using this module, we can compress a large file into smaller files in Python. The following is the process that we have to follow for compressing the large files.

• Import the Module ? Firstly, we have to import the module zipfile available in Python.

• Open the File to be Compressed ? Next we have to open the file to be zipped in the write mode.

• Specify Compression Format and Save Location ? In this step, we have to specify the compression format and the location where we have to save the compressed file.

• Close the File ? Now, in this step we have to close the file.

Syntax

The basic syntax for using the 'zipfile' module is as follows:

import zipfile
zipfile.ZipFile(file_name,'w')

Where,

• import: The keyword to include the module.

• zipfile: The name of the module

• ZipFile: The function used to create a new ZIP file.

• file_name: The desired name for the compressed file.

• 'w': Indicates that the file is opened in write mode.

Create a Text File

Let's see an example to understand the working of the zipfile module to compress the large files in Python. Let's create a sample text file that we will later compress.

We imported the zipfile module and used the ZipFile function to compress a file. opened the file in write mode and assigned it to the variable foo_zip. Finally, we closed the file and checked its type using the type function.

import zipfile
# Create a text file  
with open("file1.txt", "w") as f:  
    for i in range(10):  
        f.write("This is line %d\r\n" % (i + 1))  

Output

The following is the output of the zipfile method for compressing the file size.

zipfile.ZipFile

Compress Large Files with Python's Zipfile

Let's see another example to understand the working of the zipfile module to compress the large files in Python. Let's create a text file and then let's compress the text file. The following code can be used for working with the zipfile module.

f= open("file1.txt","w+")
for i in range(10):
   f.write("This is line %d\r\n" % (i+1))
f.close()
f=open("file1.txt", "r")
fl =f.readlines()
for x in fl:
   print(x)

Let's see the code in detail for creating the text file using Python.

• Open a file named file1.txt in write mode and assign it to the variable f.
• Use a loop to write text into the file using the write command.
• Close the file.
• Open the file again in read mode.
• Read the lines using the readlines function and assign them to the variable fl.
• Print each line using a for loop.

Output

The following is the output of the text file created using the Python programming language.

This is line 1

This is line 2

This is line 3

This is line 4

This is line 5

This is line 6

This is line 7

This is line 8

This is line 9

This is line 10

Using 'zipfile' Module With Context Manager

Let's review the text file compression generated using Python's zipfile module.

We first import the module. Then, we use the ZipFile function to compress "file1.txt" in write mode and assign it to the variable foo_zip. We apply the write command with zip_deflated as the compression type. After closing the file, we check its type using the type function.

import zipfile
foo_zip = zipfile.ZipFile('file1.txt','w')
foo_zip.write ( 'file1.txt', compress_type=zipfile.ZIP_DEFLATED )
foo_zip.close()
type(foo_zip)

Output

The following is the output of the zipfile method for compressing the file size.

zipfile.ZipFile