Qt - Write Speed - What's the fastest method to write a file in Qt?

0
votes

I already found some posts relating to Qt and writing a file with C++, but I didn't find a comparison of existing methods and a satisfying answer to the question "What's the fastest method writing a file in Qt?"...

The task: I need to write a whole bunch of double values (around 500.000 up to 1.000.000) to file to a file. Due to my program structure, this values are saved in a QList, which contains some QVectors (every QVector has same size). The QVectors contains the double values. Additionally, each column (each QList element is a column) has to be written with different precision. Each column is seperated with a \t.

I tried several methods:

  • QTextStream to a QFile with operator <<
  • Directly writing to a QFile with QFile::write
  • FILE and fwrite

In the end I got the best results with QTextStream and FILE using fwrite (nearly same speed), QFile with the QFile::write was a little bit slower (maybe 30%).

But I still think that there must be a faster method. For 500.000 double values my PC (Core2Duo) needs about 2s, and that's quite very long.

I also thought about using the Boost Karma C++ Library - can you recommend it? http://www.boost.org/doc/libs/1_51_0/libs/spirit/doc/html/spirit/karma.html

1
c++performanceqtfile
Do you absolutely need to write them as text? Converting doubles to text is a pretty significant bottleneck. – paddy
If you really care about performance, you probably should start by replacing both QList and QVector by std::vector, as QT containers are not movable. If that is not possible, you should at least replace the QList by QVector for better cache behavior. – Corristo
On a side note, you may want to consider QSaveFile – TheDarkKnight
@Corristo That's just incorrect. A QList of a type that has the alignment and size no greater than a void* has the layout and performance of a QVector. Qt containers certainly are movable - they have been so for almost 3 years, since Qt 5.2! There'll be no performance benefit to std::vector here. Zilch. – Kuba hasn't forgotten Monica
There's no such thing as "writing" values to a file: it's too generic to be meaningful. How do you want these values "written"? Is it a binary file? A text file? If text file, are you parallelizing conversion from floating point to text? And so on. Please give a complete and minimal example. – Kuba hasn't forgotten Monica

1 Answers

2
votes

Assuming you are doing it like below, it is unlikely you will find anything faster than fwrite.

double values[1000] = {...};

fwrite(values, sizeof(double), 1000, f);