What's the de-facto way of reading and writing files in Rust 1.x?

186
votes

With Rust being comparatively new, I've seen far too many ways of reading and writing files. Many are extremely messy snippets someone came up with for their blog, and 99% of the examples I've found (even on Stack Overflow) are from unstable builds that no longer work. Now that Rust is stable, what is a simple, readable, non-panicking snippet for reading or writing files?

This is the closest I've gotten to something that works in terms of reading a text file, but it's still not compiling even though I'm fairly certain I've included everything I should have. This is based off of a snippet I found on Google+ of all places, and the only thing I've changed is that the old BufferedReader is now just BufReader:

use std::fs::File;
use std::io::BufReader;
use std::path::Path;

fn main() {
    let path = Path::new("./textfile");
    let mut file = BufReader::new(File::open(&path));
    for line in file.lines() {
        println!("{}", line);
    }
}

The compiler complains:

error: the trait bound `std::result::Result<std::fs::File, std::io::Error>: std::io::Read` is not satisfied [--explain E0277]
 --> src/main.rs:7:20
  |>
7 |>     let mut file = BufReader::new(File::open(&path));
  |>                    ^^^^^^^^^^^^^^
note: required by `std::io::BufReader::new`

error: no method named `lines` found for type `std::io::BufReader<std::result::Result<std::fs::File, std::io::Error>>` in the current scope
 --> src/main.rs:8:22
  |>
8 |>     for line in file.lines() {
  |>                      ^^^^^

To sum it up, what I'm looking for is:

  • brevity
  • readability
  • covers all possible errors
  • doesn't panic
3
file-iorust
How do you want to read the file? Do you want it line-by-line, as you've shown? Do you want it all in one string? There's more than one way to "read a file". – Shepmaster
Either manner is fine. I left it open intentionally. If it's collected all into one string, splitting it into a Vec<String> would be trivial, and vice versa. At this point in my search for solutions, I'll be happy to just see elegant, up-to-date Rust file I/O code that works. – Jared
Regarding the trait error (std::io::Read), note that in Rust you must import the traits you expect to be using explicitly; thus here you are missing a use std::io::Read (which could be a use std::io::{Read,BufReader} to coalesce the two uses together) – Matthieu M.

3 Answers

260
votes

None of the functions I show here panic on their own, but I am using expect because I don't know what kind of error handling will fit best into your application. Go read The Rust Programming Language's chapter on error handling to understand how to appropriately handle failure in your own program.

Rust 1.26 and onwards

If you don't want to care about the underlying details, there are one-line functions for reading and writing.

Read a file to a String

use std::fs;

fn main() {
    let data = fs::read_to_string("/etc/hosts").expect("Unable to read file");
    println!("{}", data);
}

Read a file as a Vec<u8>

use std::fs;

fn main() {
    let data = fs::read("/etc/hosts").expect("Unable to read file");
    println!("{}", data.len());
}

Write a file

use std::fs;

fn main() {
    let data = "Some data!";
    fs::write("/tmp/foo", data).expect("Unable to write file");
}

Rust 1.0 and onwards

These forms are slightly more verbose than the one-line functions that allocate a String or Vec for you, but are more powerful in that you can reuse allocated data or append to an existing object.

Reading data

Reading a file requires two core pieces: File and Read.

Read a file to a String

use std::fs::File;
use std::io::Read;

fn main() {
    let mut data = String::new();
    let mut f = File::open("/etc/hosts").expect("Unable to open file");
    f.read_to_string(&mut data).expect("Unable to read string");
    println!("{}", data);
}

Read a file as a Vec<u8>

use std::fs::File;
use std::io::Read;

fn main() {
    let mut data = Vec::new();
    let mut f = File::open("/etc/hosts").expect("Unable to open file");
    f.read_to_end(&mut data).expect("Unable to read data");
    println!("{}", data.len());
}

Write a file

Writing a file is similar, except we use the Write trait and we always write out bytes. You can convert a String / &str to bytes with as_bytes:

use std::fs::File;
use std::io::Write;

fn main() {
    let data = "Some data!";
    let mut f = File::create("/tmp/foo").expect("Unable to create file");
    f.write_all(data.as_bytes()).expect("Unable to write data");
}

Buffered I/O

I felt a bit of a push from the community to use BufReader and BufWriter instead of reading straight from a file

A buffered reader (or writer) uses a buffer to reduce the number of I/O requests. For example, it's much more efficient to access the disk once to read 256 bytes instead of accessing the disk 256 times.

That being said, I don't believe a buffered reader/writer will be useful when reading the entire file. read_to_end seems to copy data in somewhat large chunks, so the transfer may already be naturally coalesced into fewer I/O requests.

Here's an example of using it for reading:

use std::fs::File;
use std::io::{BufReader, Read};

fn main() {
    let mut data = String::new();
    let f = File::open("/etc/hosts").expect("Unable to open file");
    let mut br = BufReader::new(f);
    br.read_to_string(&mut data).expect("Unable to read string");
    println!("{}", data);
}

And for writing:

use std::fs::File;
use std::io::{BufWriter, Write};

fn main() {
    let data = "Some data!";
    let f = File::create("/tmp/foo").expect("Unable to create file");
    let mut f = BufWriter::new(f);
    f.write_all(data.as_bytes()).expect("Unable to write data");
}

A BufReader is more useful when you want to read line-by-line:

use std::fs::File;
use std::io::{BufRead, BufReader};

fn main() {
    let f = File::open("/etc/hosts").expect("Unable to open file");
    let f = BufReader::new(f);

    for line in f.lines() {
        let line = line.expect("Unable to read line");
        println!("Line: {}", line);
    }
}
2
votes

For anybody who is writing to a file, the accepted answer is good but if you need to append to the file you have to use the OpenOptions struct instead:

use std::fs::OpenOptions;

fn main() {
    let data = "Some data!\n";
    let mut f = OpenOptions::new()
        .append(true)
        .create(true) // Optionally create the file if it doesn't already exist
        .open("/tmp/foo")
        .expect("Unable to open file");
    f.write_all(data.as_bytes()).expect("Unable to write data");
}

Buffered writing still works the same way:

use std::io::{BufWriter, Write};
use std::fs::OpenOptions;

fn main() {
    let data = "Some data!\n";
    let f = OpenOptions::new()
        .append(true)
        .open("/tmp/foo")
        .expect("Unable to open file");
    let mut f = BufWriter::new(f);
    f.write_all(data.as_bytes()).expect("Unable to write data");
}
2
votes

By using the Buffered I/O you can copy the file size is greater than the actual memory.


use std::fs::{File, OpenOptions};
use std::io::{BufReader, BufWriter, Write, BufRead};

fn main() {
    let read = File::open(r#"E:\1.xls"#);

    let write = OpenOptions::new().write(true).create(true).open(r#"E:\2.xls"#);

    let mut reader = BufReader::new(read.unwrap());

    let mut writer = BufWriter::new(write.unwrap());

    let mut length = 1;

    while length > 0 {
        let buffer = reader.fill_buf().unwrap();

        writer.write(buffer);

        length = buffer.len();
        reader.consume(length);
    }
}