1
votes

This is my first post here. I am a hobbyist so please bear with me.

I am attempting to to grab a webpage from https://eztv.it/shows/1/24/ with the following code.

public static void WriteHTMLToFile(String URL){
    try {

        URI myURI=new URI(URL);
        URL url = myURI.toURL();
        HttpsURLConnection con= (HttpsURLConnection)url.openConnection();
        File myFile=new File("c:\\project\\Test.txt");
        myFile.createNewFile();
        FileWriter wr=new FileWriter(myFile);
        InputStream ins=con.getInputStream();
        InputStreamReader isr= new InputStreamReader(ins);
        BufferedReader reader = new BufferedReader(isr);
        String line;
        while ((line = reader.readLine()) != null) {
            wr.write(line+"\n");    
        }

        reader.close();
        wr.close();
    }
    catch(Exception e){
        log(e.toString());
        }   
}

When I run this I get the following:

javax.net.ssl.SSLException: SSL peer shut down incorrectly

If I run the above code on this URL: https://eztv.it/shows/887/the-blacklist/ it works as intended. The difference between the two URL file sizes seems to be a contributing factor. In testing different URLs to the same server the above code only seemed to work for files less that ~30Kb. Anything over would generate the above exception.

1

1 Answers

2
votes

I figured it out. The server is responding with gzip encoding once file sizes are over a certain size.

con.setRequestProperty("Accept-Encoding", "gzip, deflate, sdch");

was added to the request header as well as some code to handle the gzip stream.