I've been trying to learn Go on my own, but I've been stumped on trying read from and write to ordinary files.
I can get as far as inFile, _ := os.Open(INFILE, 0, 0), but actually getting the content of the file doesn't make sense, because the read function takes a []byte as a parameter.
func (file *File) Read(b []byte) (n int, err Error)
Let's make a Go 1-compatible list of all the ways to read and write files in Go.
Because file API has changed recently and most other answers don't work with Go 1. They also miss bufio which is important IMHO.
In the following examples I copy a file by reading from it and writing to the destination file.
Start with the basics
package main
import (
"io"
"os"
)
func main() {
// open input file
fi, err := os.Open("input.txt")
if err != nil {
panic(err)
}
// close fi on exit and check for its returned error
defer func() {
if err := fi.Close(); err != nil {
panic(err)
}
}()
// open output file
fo, err := os.Create("output.txt")
if err != nil {
panic(err)
}
// close fo on exit and check for its returned error
defer func() {
if err := fo.Close(); err != nil {
panic(err)
}
}()
// make a buffer to keep chunks that are read
buf := make([]byte, 1024)
for {
// read a chunk
n, err := fi.Read(buf)
if err != nil && err != io.EOF {
panic(err)
}
if n == 0 {
break
}
// write a chunk
if _, err := fo.Write(buf[:n]); err != nil {
panic(err)
}
}
}
Here I used os.Open and os.Create which are convenient wrappers around os.OpenFile. We usually don't need to call OpenFile directly.
Notice treating EOF. Read tries to fill buf on each call, and returns io.EOF as error if it reaches end of file in doing so. In this case buf will still hold data. Consequent calls to Read returns zero as the number of bytes read and same io.EOF as error. Any other error will lead to a panic.
Using bufio
package main
import (
"bufio"
"io"
"os"
)
func main() {
// open input file
fi, err := os.Open("input.txt")
if err != nil {
panic(err)
}
// close fi on exit and check for its returned error
defer func() {
if err := fi.Close(); err != nil {
panic(err)
}
}()
// make a read buffer
r := bufio.NewReader(fi)
// open output file
fo, err := os.Create("output.txt")
if err != nil {
panic(err)
}
// close fo on exit and check for its returned error
defer func() {
if err := fo.Close(); err != nil {
panic(err)
}
}()
// make a write buffer
w := bufio.NewWriter(fo)
// make a buffer to keep chunks that are read
buf := make([]byte, 1024)
for {
// read a chunk
n, err := r.Read(buf)
if err != nil && err != io.EOF {
panic(err)
}
if n == 0 {
break
}
// write a chunk
if _, err := w.Write(buf[:n]); err != nil {
panic(err)
}
}
if err = w.Flush(); err != nil {
panic(err)
}
}
bufio is just acting as a buffer here, because we don't have much to do with data. In most other situations (specially with text files) bufio is very useful by giving us a nice API for reading and writing easily and flexibly, while it handles buffering behind the scenes.
Note: The following code is for older Go versions (Go 1.15 and before). Things have changed. For the new way, take a look at this answer.
Using ioutil
package main
import (
"io/ioutil"
)
func main() {
// read the whole file at once
b, err := ioutil.ReadFile("input.txt")
if err != nil {
panic(err)
}
// write the whole body at once
err = ioutil.WriteFile("output.txt", b, 0644)
if err != nil {
panic(err)
}
}
Easy as pie! But use it only if you're sure you're not dealing with big files.
Using io.Copy
package main
import (
"io"
"log"
"os"
)
func main () {
// open files r and w
r, err := os.Open("input.txt")
if err != nil {
panic(err)
}
defer r.Close()
w, err := os.Create("output.txt")
if err != nil {
panic(err)
}
defer w.Close()
// do the actual work
n, err := io.Copy(w, r)
if err != nil {
panic(err)
}
log.Printf("Copied %v bytes\n", n)
}
If you don't feel like reinventing the wheel, the io.Copy and io.CopyN may serve you well. If you check the source of the io.Copy function, it is nothing but one of the Mostafa's solutions (the 'basic' one, actually) packaged in the Go library. They are using a significantly larger buffer than he is, though.
With newer Go versions, reading/writing to/from a file is easy. To read from a file:
package main
import (
"fmt"
"io/ioutil"
)
func main() {
data, err := ioutil.ReadFile("text.txt")
if err != nil {
return
}
fmt.Println(string(data))
}
To write to a file:
package main
import "os"
func main() {
file, err := os.Create("text.txt")
if err != nil {
return
}
defer file.Close()
file.WriteString("test\nhello")
}
This will overwrite the content of a file (create a new file if it was not there).
[]byte is a slice (similar to a substring) of all or part of a byte array. Think of the slice as a value structure with a hidden pointer field for the system to locate and access all or part of an array (the slice), plus fields for the length and capacity of the slice, which you can access using the len() and cap() functions.
Here's a working starter kit for you, which reads and prints a binary file; you will need to change the inName literal value to refer to a small file on your system.
package main
import (
"fmt";
"os";
)
func main()
{
inName := "file-rw.bin";
inPerm := 0666;
inFile, inErr := os.Open(inName, os.O_RDONLY, inPerm);
if inErr == nil {
inBufLen := 16;
inBuf := make([]byte, inBufLen);
n, inErr := inFile.Read(inBuf);
for inErr == nil {
fmt.Println(n, inBuf[0:n]);
n, inErr = inFile.Read(inBuf);
}
}
inErr = inFile.Close();
}
Starting with Go 1.16, use os.ReadFile to load the file to memory, and use os.WriteFile to write to a file from memory.
Be careful with the os.ReadFile because it reads the whole file into memory.
package main
import "os"
func main() {
b, err := os.ReadFile("input.txt")
if err != nil {
log.Fatal(err)
}
// `data` contains everything your file does
// This writes it to the Standard Out
os.Stdout.Write(data)
// You can also write it to a file as a whole
err = os.WriteFile("destination.txt", b, 0644)
if err != nil {
log.Fatal(err)
}
}
Just looking at the documentation it seems you should just declare a buffer of type []byte and pass it to read which will then read up to that many characters and return the number of characters actually read (and an error).
The docs say
Read reads up to len(b) bytes from the File. It returns the number of bytes read and an Error, if any. EOF is signaled by a zero count with err set to EOF.
Does that not work?
EDIT: Also, I think you should perhaps use the Reader/Writer interfaces declared in the bufio package instead of using os package.
The Read method takes a byte parameter because that is the buffer it will read into. It's a common idiom in some circles and makes some sense when you think about it.
This way you can determine how many bytes will be read by the reader and inspect the return to see how many bytes actually were read and handle any errors appropriately.
As others have pointed in their answers, bufio is probably what you want for reading from most files.
I'll add one other hint since it's really useful. Reading a line from a file is best accomplished not by the ReadLine method, but the ReadBytes or ReadString method instead.
