How do I read an entire file into a std::string in C++?

Question

How do I read a file into a std::string, i.e., read the whole file at once?

Text or binary mode should be specified by the caller. The solution should be standard-compliant, portable and efficient. It should not needlessly copy the string's data, and it should avoid reallocations of memory while reading the string.

One way to do this would be to stat the filesize, resize the std::string and fread() into the std::string's const_cast<char*>()'ed data(). This requires the std::string's data to be contiguous which is not required by the standard, but it appears to be the case for all known implementations. What is worse, if the file is read in text mode, the std::string's size may not equal the file's size.

A fully correct, standard-compliant and portable solutions could be constructed using std::ifstream's rdbuf() into a std::ostringstream and from there into a std::string. However, this could copy the string data and/or needlessly reallocate memory.

• Are all relevant standard library implementations smart enough to avoid all unnecessary overhead?
• Is there another way to do it?
• Did I miss some hidden Boost function that already provides the desired functionality?

void slurp(std::string& data, bool is_binary)

Answer 1

One way is to flush the stream buffer into a separate memory stream, and then convert that to std::string:

std::string slurp(std::ifstream& in) {
    std::ostringstream sstr;
    sstr << in.rdbuf();
    return sstr.str();
}

This is nicely concise. However, as noted in the question this performs a redundant copy and unfortunately there is fundamentally no way of eliding this copy.

The only real solution that avoids redundant copies is to do the reading manually in a loop, unfortunately. Since C++ now has guaranteed contiguous strings, one could write the following (≥C++14):

auto read_file(std::string_view path) -> std::string {
    constexpr auto read_size = std::size_t{4096};
    auto stream = std::ifstream{path.data()};
    stream.exceptions(std::ios_base::badbit);

    auto out = std::string{};
    auto buf = std::string(read_size, '\0');
    while (stream.read(& buf[0], read_size)) {
        out.append(buf, 0, stream.gcount());
    }
    out.append(buf, 0, stream.gcount());
    return out;
}

Answer 2

The shortest variant: Live On Coliru

std::string str(std::istreambuf_iterator<char>{ifs}, {});

It requires the header <iterator>.

There were some reports that this method is slower than preallocating the string and using std::istream::read. However, on a modern compiler with optimisations enabled this no longer seems to be the case, though the relative performance of various methods seems to be highly compiler dependent.

Answer 3

See this answer on a similar question.

For your convenience, I'm reposting CTT's solution:

string readFile2(const string &fileName)
{
    ifstream ifs(fileName.c_str(), ios::in | ios::binary | ios::ate);

    ifstream::pos_type fileSize = ifs.tellg();
    ifs.seekg(0, ios::beg);

    vector<char> bytes(fileSize);
    ifs.read(bytes.data(), fileSize);

    return string(bytes.data(), fileSize);
}

This solution resulted in about 20% faster execution times than the other answers presented here, when taking the average of 100 runs against the text of Moby Dick (1.3M). Not bad for a portable C++ solution, I would like to see the results of mmap'ing the file ;)

Answer 4

If you have C++17 (std::filesystem), there is also this way (which gets the file's size through std::filesystem::file_size instead of seekg and tellg):

#include <filesystem>
#include <fstream>
#include <string>

namespace fs = std::filesystem;

std::string readFile(fs::path path)
{
    // Open the stream to 'lock' the file.
    std::ifstream f(path, std::ios::in | std::ios::binary);

    // Obtain the size of the file.
    const auto sz = fs::file_size(path);

    // Create a buffer.
    std::string result(sz, '\0');

    // Read the whole file into the buffer.
    f.read(result.data(), sz);

    return result;
}

Note: you may need to use <experimental/filesystem> and std::experimental::filesystem if your standard library doesn't yet fully support C++17. You might also need to replace result.data() with &result[0] if it doesn't support non-const std::basic_string data.

Answer 5

Use

#include <iostream>
#include <sstream>
#include <fstream>

int main()
{
  std::ifstream input("file.txt");
  std::stringstream sstr;

  while(input >> sstr.rdbuf());

  std::cout << sstr.str() << std::endl;
}

or something very close. I don't have a stdlib reference open to double-check myself.

Yes, I understand I didn't write the slurp function as asked.

Answer 6

I do not have enough reputation to comment directly on responses using tellg().

Please be aware that tellg() can return -1 on error. If you're passing the result of tellg() as an allocation parameter, you should sanity check the result first.

An example of the problem:

...
std::streamsize size = file.tellg();
std::vector<char> buffer(size);
...

In the above example, if tellg() encounters an error it will return -1. Implicit casting between signed (ie the result of tellg()) and unsigned (ie the arg to the vector<char> constructor) will result in a your vector erroneously allocating a very large number of bytes. (Probably 4294967295 bytes, or 4GB.)

Modifying paxos1977's answer to account for the above:

string readFile2(const string &fileName)
{
    ifstream ifs(fileName.c_str(), ios::in | ios::binary | ios::ate);

    ifstream::pos_type fileSize = ifs.tellg();
    if (fileSize < 0)                             <--- ADDED
        return std::string();                     <--- ADDED

    ifs.seekg(0, ios::beg);

    vector<char> bytes(fileSize);
    ifs.read(&bytes[0], fileSize);

    return string(&bytes[0], fileSize);
}

Answer 7

This solution adds error checking to the rdbuf()-based method.

std::string file_to_string(const std::string& file_name)
{
    std::ifstream file_stream{file_name};

    if (file_stream.fail())
    {
        // Error opening file.
    }

    std::ostringstream str_stream{};
    file_stream >> str_stream.rdbuf();  // NOT str_stream << file_stream.rdbuf()

    if (file_stream.fail() && !file_stream.eof())
    {
        // Error reading file.
    }

    return str_stream.str();
}

I'm adding this answer because adding error-checking to the original method is not as trivial as you'd expect. The original method uses stringstream's insertion operator (str_stream << file_stream.rdbuf()). The problem is that this sets the stringstream's failbit when no characters are inserted. That can be due to an error or it can be due to the file being empty. If you check for failures by inspecting the failbit, you'll encounter a false positive when you read an empty file. How do you disambiguate legitimate failure to insert any characters and "failure" to insert any characters because the file is empty?

You might think to explicitly check for an empty file, but that's more code and associated error checking.

Checking for the failure condition str_stream.fail() && !str_stream.eof() doesn't work, because the insertion operation doesn't set the eofbit (on the ostringstream nor the ifstream).

So, the solution is to change the operation. Instead of using ostringstream's insertion operator (<<), use ifstream's extraction operator (>>), which does set the eofbit. Then check for the failiure condition file_stream.fail() && !file_stream.eof().

Importantly, when file_stream >> str_stream.rdbuf() encounters a legitimate failure, it shouldn't ever set eofbit (according to my understanding of the specification). That means the above check is sufficient to detect legitimate failures.

Answer 8

Something like this shouldn't be too bad:

void slurp(std::string& data, const std::string& filename, bool is_binary)
{
    std::ios_base::openmode openmode = ios::ate | ios::in;
    if (is_binary)
        openmode |= ios::binary;
    ifstream file(filename.c_str(), openmode);
    data.clear();
    data.reserve(file.tellg());
    file.seekg(0, ios::beg);
    data.append(istreambuf_iterator<char>(file.rdbuf()), 
                istreambuf_iterator<char>());
}

The advantage here is that we do the reserve first so we won't have to grow the string as we read things in. The disadvantage is that we do it char by char. A smarter version could grab the whole read buf and then call underflow.

Answer 9

Here's a version using the new filesystem library with reasonably robust error checking:

#include <cstdint>
#include <exception>
#include <filesystem>
#include <fstream>
#include <sstream>
#include <string>

namespace fs = std::filesystem;

std::string loadFile(const char *const name);
std::string loadFile(const std::string &name);

std::string loadFile(const char *const name) {
  fs::path filepath(fs::absolute(fs::path(name)));

  std::uintmax_t fsize;

  if (fs::exists(filepath)) {
    fsize = fs::file_size(filepath);
  } else {
    throw(std::invalid_argument("File not found: " + filepath.string()));
  }

  std::ifstream infile;
  infile.exceptions(std::ifstream::failbit | std::ifstream::badbit);
  try {
    infile.open(filepath.c_str(), std::ios::in | std::ifstream::binary);
  } catch (...) {
    std::throw_with_nested(std::runtime_error("Can't open input file " + filepath.string()));
  }

  std::string fileStr;

  try {
    fileStr.resize(fsize);
  } catch (...) {
    std::stringstream err;
    err << "Can't resize to " << fsize << " bytes";
    std::throw_with_nested(std::runtime_error(err.str()));
  }

  infile.read(fileStr.data(), fsize);
  infile.close();

  return fileStr;
}

std::string loadFile(const std::string &name) { return loadFile(name.c_str()); };

Answer 10

Since this seems like a widely used utility, my approach would be to search for and to prefer already available libraries to hand made solutions, especially if boost libraries are already linked(linker flags -lboost_system -lboost_filesystem) in your project. Here (and older boost versions too), boost provides a load_string_file utility:

#include <iostream>
#include <string>
#include <boost/filesystem/string_file.hpp>

int main() {
    std::string result;
    boost::filesystem::load_string_file("aFileName.xyz", result);
    std::cout << result.size() << std::endl;
}

As an advantage, this function doesn't seek an entire file to determine the size, instead uses stat() internally. As a possibly negligible disadvantage though, one could easily infer upon inspection of the source code: string is unnecessarily resized with '\0' character which are rewritten by the file contents.

Answer 11

You can use the 'std::getline' function, and specify 'eof' as the delimiter. The resulting code is a little bit obscure though:

std::string data;
std::ifstream in( "test.txt" );
std::getline( in, data, std::string::traits_type::to_char_type( 
                  std::string::traits_type::eof() ) );

Answer 12

#include <string>
#include <sstream>

using namespace std;

string GetStreamAsString(const istream& in)
{
    stringstream out;
    out << in.rdbuf();
    return out.str();
}

string GetFileAsString(static string& filePath)
{
    ifstream stream;
    try
    {
        // Set to throw on failure
        stream.exceptions(fstream::failbit | fstream::badbit);
        stream.open(filePath);
    }
    catch (system_error& error)
    {
        cerr << "Failed to open '" << filePath << "'\n" << error.code().message() << endl;
        return "Open fail";
    }

    return GetStreamAsString(stream);
}

usage:

const string logAsString = GetFileAsString(logFilePath);

Answer 13

An updated function which builds upon CTT's solution:

#include <string>
#include <fstream>
#include <limits>
#include <string_view>
std::string readfile(const std::string_view path, bool binaryMode = true)
{
    std::ios::openmode openmode = std::ios::in;
    if(binaryMode)
    {
        openmode |= std::ios::binary;
    }
    std::ifstream ifs(path.data(), openmode);
    ifs.ignore(std::numeric_limits<std::streamsize>::max());
    std::string data(ifs.gcount(), 0);
    ifs.seekg(0);
    ifs.read(data.data(), data.size());
    return data;
}

There are two important differences:

tellg() is not guaranteed to return the offset in bytes since the beginning of the file. Instead, as Puzomor Croatia pointed out, it's more of a token which can be used within the fstream calls. gcount() however does return the amount of unformatted bytes last extracted. We therefore open the file, extract and discard all of its contents with ignore() to get the size of the file, and construct the output string based on that.

Secondly, we avoid having to copy the data of the file from a std::vector<char> to a std::string by writing to the string directly.

In terms of performance, this should be the absolute fastest, allocating the appropriate sized string ahead of time and calling read() once. As an interesting fact, using ignore() and countg() instead of ate and tellg() on gcc compiles down to almost the same thing, bit by bit.

Answer 14

Never write into the std::string's const char * buffer. Never ever! Doing so is a massive mistake.

Reserve() space for the whole string in your std::string, read chunks from your file of reasonable size into a buffer, and append() it. How large the chunks have to be depends on your input file size. I'm pretty sure all other portable and STL-compliant mechanisms will do the same (yet may look prettier).

Answer 15

#include <iostream>
#include <fstream>
#include <string.h>
using namespace std;
main(){
    fstream file;
    //Open a file
    file.open("test.txt");
    string copy,temp;
    //While loop to store whole document in copy string
    //Temp reads a complete line
    //Loop stops until temp reads the last line of document
    while(getline(file,temp)){
        //add new line text in copy
        copy+=temp;
        //adds a new line
        copy+="\n";
    }
    //Display whole document
    cout<<copy;
    //close the document
    file.close();
}