Boost::mutex is taking less time than without mutex for a program

Question

I have executed below program where I have created 100 threads to execute concurrently. Please note this is a sample program made. I understand multiple threads are not required for below program but my intention was to test mutex.

class ThreadPool{

    public:
    ThreadPool(int num = 10);
    ~ThreadPool();
    void AssignPool();
    void doSometask();
    void inc();
    private:
    boost::asio::io_service ioService;
    boost::thread_group threadpool;
    boost::asio::io_service::work * work;
   volatile int p_size;
    int pool_sz;
    boost::mutex io_mutex;// with boost lock

};

void ThreadPool::AssignPool()
{
        std::cout<<std::endl<<"pool_sz="<<pool_sz<<std::endl;
        for(int i=0;i<pool_sz;i++)
        {
                ioService.post(boost::bind(&ThreadPool::doSometask, this));
        }
}

void ThreadPool::inc()
{
        p_size++;
}

void ThreadPool::doSometask()
{

//      boost::mutex::scoped_lock lock(io_mutex);
        for(int i=0;i<10000;i++){
                inc();
        }


}

ThreadPool::ThreadPool(int num):p_size(0)
{
        pool_sz = num;
        work = new  boost::asio::io_service::work(ioService);
        for(int i =0;i<num;i++)
        {
                threadpool.create_thread(boost::bind(&boost::asio::io_service::run, &ioService  ))    ;
        }

}

ThreadPool::~ThreadPool()
{
        delete work;
        ioService.stop();
        threadpool.join_all();
}

int main()
{
        ThreadPool p1(100);
        p1.AssignPool();
}

Case 1: Above program was executed by commenting "boost::mutex::scoped_lock lock(io_mutex);" line which is "no mutex case". time taken by the program was

real    0m1.386s
user    0m0.483s
sys 0m9.937s

Case 2: With Mutex: However when I run this program with mutex i.e. "boost::mutex::scoped_lock lock(io_mutex);" line. This program is taking lesser time.

real    0m0.289s
user    0m0.067s
sys 0m0.230s

In my understanding with mutex, program should have taken much more time than without mutex. What went wrong here??

100 threads trying to increment the same variable will likely cause a serious cache thrashing on a multi core system. Sometimes "one at a time in an orderly fashion" works better than everyone just rushing in. — Bo Persson
I understand your point, could you let me know the way to test mutex performance? I want to see the difference between mutex and "no mutex" case for 100 threads. Is there a way to see real difference? — SACHIN GOYAL
100 threads is also really nonsense, unless you are running on a 100+ CPU system. — roalz
"In my understanding with mutex, program should have taken much more time than without mutex." Your understanding is wrong. Mutexes prevent contending threads from being scheduled concurrently, typically significantly improving performance over schemes that tolerate high contention. That's why they're the go-to form of synchronization for typical cases. — David Schwartz

Gert Wollny Gert Wollny · Accepted Answer · 2017-04-06T11:39:33

In your example you lock the mutex in doSometask(), and, hence all the time only one thread will be running and it will finish the for loop before yielding to another task. Hence the program runs literally serial and no cache threshing occurs.

Without the lock, all threads will run when they get processor time, and assuming that the number of processors is significantly lower than 100 a lot of cache threshing will be going on on all levels (like Bo Persson wrote in the comments), and this will increase run-time.

A better way to measure the impact of the lock on run time would be (a) to run only as many threads as your computer has cores, so that cache threshing because of context switches are minimized, and (b), to put the lock into the ThreadPool::inc() method so that the synchronization happens more often.

As a bonus you could run the lock-free method properly by declaring p_size as std::atomic<int> (C++11) and see impact of synchronization based on mutexes versus the use of atomics.

Boost::mutex is taking less time than without mutex for a program

3 Answers