When does concurrency/multithreading help improve performance?

Question

I have been planning to use concurrency in project after learning it indeed has increased through put for many. Now I have not worked much on multi threading or concurrency so decided to learn and have a simple proof of concept before using it in actual project.
Below are the two examples I have tried:

1. With use of concurrency

    public static void main(String[] args) 

    {
        System.out.println("start main ");

        ExecutorService es = Executors.newFixedThreadPool(3);
        long startTime = new Date().getTime();
        Collection<SomeComputation> collection = new ArrayList<SomeComputation>();
        for(int i=0; i< 10000; i++){
            collection.add(new SomeComputation("SomeComputation"+i));
        }
        try {
            List<Future< Boolean >> list = es.invokeAll(collection);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println("\n end main "+(new Date().getTime() - startTime));
    }

2. Without use of concurrency

        public static void main(String[] args) {

          System.out.println("start main ");
          long startTime = new Date().getTime();
          Collection<SomeComputation> collection = new ArrayList<SomeComputation>();
          for(int i=0; i< 10000; i++){
          collection.add(new SomeComputation("SomeComputation"+i));
        }
        for(SomeComputation sc:collection)
        {
        sc.compute();
        }
        System.out.println("\n end main "+(new Date().getTime() - startTime));
        }

    Both share a common class

        class SomeComputation implements Callable<Boolean>
        {
            String name;
            SomeComputation(String name){this.name=name;}
            public Boolean compute()
            {
                someDumbStuff();
                return true;
            }

            public Boolean call()
            {
                someDumbStuff();
                return true;
            }

            private void someDumbStuff()
            {
                for (int i = 0;i<50000;i++)
                {
                    Integer.compare(i,i+1);
                }
                System.out.print("\n done with "+this.name);
            }
        }

Now the analysis after 20 odd runs of each approach.
1st one with concurrency takes on average 451 msecs.
2nd one without concurrency takes on average 290 msecs.
Now I learned this depends on configuration , OS , version(java 7) and processor. But all was same for both approaches. Also learned the cost of concurrency is affordable on when computation is heavy.But this point wasn't clear to me.
Hope some one can help me understand this more.
PS: I tried finding similar questions but could find this kind.Please comment the link if you do.

The late Gene Amdahl did some research you may find helpful; see also Amdahl's law. — Elliott Frisch
Try increasing the loop duration in someDumbStuff to about 1000000 and you may start to see a more expected trend. — flakes
@flkes yes.. if I do that the approach with concurrency takes half the time compared to approach without concurrency.. this has something to learn.. thanks.. — Viraj Nalawade

Mike Dunlavey Mike Dunlavey · Accepted Answer · 2016-03-24T12:28:28

Concurrency has at least two different purposes: 1) performance, and 2) simplicity of code (like 1000 listeners for web requests).

If your purpose is performance, you can't get any more speedup than the number of hardware cores you put to work. (And that's only if the threads are CPU-bound.) What's more, each thread has a significant startup overhead. So if you launch 1000 threads on a 4-core machine, you can't possibly do any better than a 4x speedup, but against that, you have 1000 thread startup costs.

When does concurrency/multithreading help improve performance?

3 Answers