Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
Key Steps in Read/Write from/to a URL Connection in Java
The URL class in the java.net package represents a web address or URL (Uniform Resource Locator), which points to a web resource such as a file, page, or directory in the World Wide Web (internet).
The URL class provides various constructors, one of which accepts a String parameter (representing a URL) and constructs an object of the URL class.
Reading From a URL
You can read content from a URL using the openStream() method. This method opens a connection to the web address specified by the current URL object and gives you an InputStream object.
The Java InputStream is an abstract class, and it is the super class of all classes that represent an input stream of bytes. You can read the contents from an InputStream object (in this case, data from the URL) using classes like Scanner.
Steps to Read Data From a URL
The following are the key steps to read data from a URL (using the Java URL class) -
-
Step 1: Instantiate the java.net.URL class by passing the URL of the desired web page as a parameter to its constructor.
-
Step 2: Invoke the openStream() method and retrieve the InputStream object.
-
Step 3: Instantiate the Scanner class by passing the InputStream object (retrieved above) as a parameter.
Example
The following example reads the content of "http://www.something.com/" and prints it -
import java.io.IOException;
import java.net.URL;
import java.util.Scanner;
import java.io.InputStream;
public class ReadingWebPage {
public static void main(String args[]) throws IOException {
try {
//Instantiating the URL class
URL url = new URL("http://www.something.com/");
//Retrieving the contents of the specified page
InputStream inputStream = url.openStream();
Scanner sc = new Scanner(inputStream);
//Instantiating the StringBuffer class to hold the result
StringBuffer sb = new StringBuffer();
while(sc.hasNext()) {
sb.append(sc.next());
//System.out.println(sc.next());
}
//Retrieving the String from the String Buffer object
String result = sb.toString();
System.out.println(result);
//Removing the HTML tags
result = result.replaceAll("<[^>]*>", "");
System.out.println("Contents of the web page: "+result);
} catch (Exception e) {
e.printStackTrace();
}
}
}
Following is the output of the above program -
<html><body><h1>Itworks!</h1></body></html> Contents of the web page: Itworks!
Writing data to a URL
The URL class provides a method named openConnection(), which opens a connection with the current URL and returns an instance of it.
To send data, you need to configure your request accordingly, which means setting the request method as "POST" and setting the value for the DoOutput field as true using the setDoOutput method.
Steps to Write Data to a URL
The following are the key steps to read data from a URL (using the Java URL class) -
-
Step 1: Instantiate the java.net.URL class by passing the URL of the desired web page as a parameter to its constructor.
-
Step 2: Invoke the openConnection() method to open the connection.
-
Step 3: Configure the request through the obtained connection.
-
Step 4: Get the OutputStream from the connection (object of HttpURLConnection), write the data to it using the write() method.
Example
The following example writes data to the URL https://httpbin.org/post -
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.URL;
public class WritingToWebPage {
public static void main(String[] args) {
try {
//Instantiating the URL class
URL url = new URL("https://httpbin.org/post");
//Opening the connection
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
// Configure the request
conn.setRequestMethod("POST");
conn.setDoOutput(true);
// Write data to the request body
try (OutputStream os = conn.getOutputStream()) {
byte[] data = "test data".getBytes("utf-8");
os.write(data, 0, data.length);
}
System.out.println("Response: " + conn.getResponseMessage());
} catch (Exception e) {
e.printStackTrace();
}
}
}
Following is the output of the above program -
Response: OK
Exceptions When Reading/Writing from a URL in Java
When working with the URL and openstream() methods, sometimes it might throw some exceptions at runtime. The Exceptions that can occur are -
-
MalformedURLException
-
IOException
MalformedURLException
It occurs if the URL provided by you is not valid. For example, if the URL is missing a protocol(http://, https://), or is wrongly formatted.
import java.net.URL;
import java.util.Scanner;
public class MalformedURLDemo {
public static void main(String[] args) throws Exception {
// Invalid URL ? missing protocol (http://)
URL url = new URL("www.invalid-url.com"); // This will throw MalformedURLException
Scanner sc = new Scanner(url.openStream());
while (sc.hasNext()) {
System.out.println(sc.next());
}
}
}
Let us compile and run the above program, which will give the following result -
1 warning ERROR! Exception in thread "main" java.net.MalformedURLException: no protocol: www.invalid-url.com at java.base/java.net.URL.<init>(URL.java:772) at java.base/java.net.URL.<init>>(URL.java:654) at java.base/java.net.URL.<init>(URL.java:590) at MalformedURLDemo.main(Main.java:7)
IOException
It occurs if there is an issue during opening the stream or reading/writing data from a URL, like(no internet connection or the server is not responding).
import java.net.URL;
import java.util.Scanner;
public class IOExceptionDemo {
public static void main(String[] args) throws Exception {
// Valid format but domain doesn't exist
URL url = new URL("http://www.nonexistentdomainforsure12345.com/");
Scanner sc = new Scanner(url.openStream()); // IOException likely here
while (sc.hasNext()) {
System.out.println(sc.next());
}
}
}
Let us compile and run the above program, which will give the following result ?
1 warning ERROR! Exception in thread "main" java.net.UnknownHostException: www.nonexistentdomainforsure12345.com at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:567) at java.base/java.net.Socket.connect(Socket.java:751) at java.base/java.net.Socket.connect(Socket.java:686) at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:183) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:531) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:636) at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:280) at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:386) at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:408) at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1319) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1252) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1138) at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1067) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1690) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1614) at java.base/java.net.URL.openStream(URL.java:1325) at IOExceptionDemo.main(Main.java:8)
Handling Exceptions
To handle these exceptions, we can use a Try-Catch block.
The try-catch block helps catch exceptions like MalformedURLException and IOException during URL connection or data reading. And it just shows an error message.
import java.net.URL;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.Scanner;
public class SafeURLReader {
public static void main(String[] args) {
try {
// Creating URL object
URL url = new URL("http://www.something.com/");
// Reading from URL
Scanner sc = new Scanner(url.openStream());
StringBuilder sb = new StringBuilder();
while (sc.hasNext()) {
sb.append(sc.next());
}
String result = sb.toString();
System.out.println("Raw content: " + result);
// Removing HTML tags
result = result.replaceAll("<[^>]*>", "");
System.out.println("Clean content: " + result);
} catch (MalformedURLException e) {
System.out.println("The URL is not formatted correctly: " + e.getMessage());
} catch (IOException e) {
System.out.println("An I/O error occurred: " + e.getMessage());
}
}
}
Following is the output of the above program -Raw content: <html><body><h1>Itworks!</h1></body></html> Clean content: Itworks!
286 Views
