Java : URLConnection setRequestProperty range issue relying on multithreaded process?

Question

I'm stuck trying to read by range a remote file.

According following trace sounds setRequestProperty is not performing its job (InputStream Length is nok to me except one .. the first one ???) ?

Task Runnable ID = 0 --> Requested Range Property = {Range=[bytes=0-93969]}

Task Runnable ID = 1 --> Resquested Range Property = {Range=[bytes=93970-187939]}

Task Runnable ID = 2 --> Requested Range Property = {Range=[bytes=187940-281909]}

Task Runnable ID = 3 --> Resquested Range Property = {Range=[bytes=281910-375883]}

Task Runnable ID = 0 --> InputStream Lenght = 93970 / StartByte = 0 / CurrentByte = 0 / EndByte = 93969

Task Runnable ID = 1 --> InputStream Length = 375883 / StartByte = 93970 / CurrentByte = 93970 / EndByte = 187939

Task Runnable ID = 3 --> InputStream Length = 375883 / StartByte = 281910 / CurrentByte = 281910 / EndByte = 375883

Task Runnable ID = 2 --> InputStream Length = 375883 / StartByte = 187940 / CurrentByte = 187940 / EndByte = 281909

My code is :

public class TaskRunnable implements Runnable {

    private static final int    BUFFER_SIZE     = 4092;

    private long                startByte;
    private long                currentByte;
    private long                endByte;


    private Task                task;
    private static int          idCounter       = 0;
    private int                 id;

    @SuppressWarnings("unused")
    private TaskRunnable() {
    }

    public TaskRunnable(Task task, long startByte, long endByte) {
        this.startByte = startByte;
        this.endByte = endByte;
        this.task = task;
        this.currentByte = startByte;
        this.id = idCounter++;
    }

    @Override
    public void run() {
        Thread.currentThread().setName("Download Runnable");

        Authenticator authenticator = task.getManager().getAuthenticator();
        if (authenticator != null) {
            Authenticator.setDefault(authenticator);
        }

        File targetFile;
        synchronized (this) {
            targetFile = new File(task.getTargetFile().getAbsolutePath());
        }

        BufferedInputStream bufferedInputStream = null;
        byte[] buf = new byte[BUFFER_SIZE];
        URLConnection urlConnection = null;
        try {
            URL _url = new URL(task.getSourceFileUrl().toString());
            Proxy proxy = task.getManager().getProxy();
            if (proxy != null) {
                urlConnection = _url.openConnection(proxy);
            } else {
                urlConnection = _url.openConnection();
            }
            urlConnection.setRequestProperty("Range", "bytes=" + currentByte + "-" + endByte);
            System.out.println("Task Runnable ID = " + id + " --> Requested Range Property = " + urlConnection.getRequestProperties().toString());

            bufferedInputStream = new BufferedInputStream(urlConnection.getInputStream());
            int len = 0;
            while (bufferedInputStream.read() != -1) {
                len++;
            }
            System.out.println("Task Runnable ID = " + id + " --> InputStream Length = " + len + " / StartByte = " + startByte + " / CurrentByte = " + currentByte + " / EndByte = " + endByte);

            bufferedInputStream.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

Obviously it's my fault but not able to figure out what's wrong. Help more than welcome. Thks !

Note : all is running fine if using same code but single thread based.

What exactly is the point all this? Why are you measuring the length of a range when you already know the range? And what exactly is the purpose of multithreading it? — user207421
Obviously I'm doing more than getting length within real life. I'm using InpuStream to get my remote file bytes. InputStream parts are written to a local file using RandomAccessFile. Current issue is not able to get correct (well sized) InputStream. — cartu38 opendev

Little Santi Little Santi · Accepted Answer · 2016-10-10T12:36:30

I suspect a concurrent modification of the internal URLConnection state. Have you tried to invoke URLConnection.setUseCaches(false) ?

Update

Another global cache that affects a possible reusing of URLConnection objects is ResponseCache. You should disable it at initialization by executing

ResponseCache.setDefault(null);

Java : URLConnection setRequestProperty range issue relying on multithreaded process?

1 Answers