Tuesday, February 12, 2019


Go - appending []byte as a variadic parameter  

In my last post, I demonstrated a type that implemented the io.Writer interface.  During the PR process for that code, one of my team members asked,  why did you use that odd syntax with append?


// newData is of type []byte
append(newData, []byte(",")...)

append() is a Go built in.  It's implemation is found here: builtins.go  There are lots of posts that talk about how append works, so I won't go into that here.  I just want to focus on answering the practical question my team member asked: why did I use that odd syntax with the "..."?

The simple answer is: that's the only way to append a slice of bytes to a slice of bytes.  Not really a helpful answer though, so here are some more details.   append() is a variadic func.  In our example, the first param is []byte.  It's a slice of "bytes".  That means that append() will only accept a byte as its second parameter.  Yep, a single byte, not a slice of bytes.  That's where the power of Go's variadic functions comes to play.  You can pass in an arbitrary number of single bytes as the second parameter to append().  Now we just need to convert []byte to multiple single bytes.  You do this by simply making []byte a variadic parameter by adding "..."

Add all of this together and you'll understand the answer and why this works:

b2 := append([]byte("hi mom"), []byte(" and dad")...)  

Here's some example code you can play with: https://play.golang.org/p/Ml5pRwryaQc

package main

import (
 "bytes"
 "strings"
 "fmt"
)


// I implemented an io.Write interface in our library and I got a question about why I used
// an odd syntax with append().  In our use case, we need to chop off the newline from
// the data param and add a comma during the Write()
//
// here's the method in question:
//  Write - simply append to the strings.Buffer but add a comma too
// func (b *LogBuffer) Write(data []byte) (n int, err error) {
// newData := bytes.TrimSuffix(data, []byte("\n"))
// return b.Buff.Write(append(newData, []byte(",")...))
// }
// taken from: https://github.com/Bose/go-gin-logrus/blob/7cf3281c11105460c98f821872b64aebc89e1b06/logBuffer.go#L17
//

func main() {
 // first some setup
 var b strings.Builder
 
 // the param to our func and it's a []byte (aka a slice)
 data := []byte("hi mom.\n") 
 
 // hack off the newline since newlines will cause problems in our use case
 newData := bytes.TrimSuffix(data, []byte("\n")) 
 
 
 // First, can't we just append the comma as a string?
 // nope, this causes an error: cannot use "," (type string) as type byte in append
 // yep, you can't append a string to []byte.  that makes sense.
 // b.Write(append(newData, ","))
 
 // Next, can't we just append the comma as []byte without any trailing ... ?
 // nope, this causes an error: cannot use ([]byte)(",") (type []byte) as type byte in append
 // What the heck?  What does this error mean?  Well, newData is a slice of bytes, so append insists
 // that you can only append a byte (or many individual bytes, since it's a variadic func) to it, 
 // not a slice of bytes (aka []byte)
 // b.Write(append(newData, []byte(",")))
 
 // Add the ... after the slice of bytes which tells Go to "unpack" the []bytes into individual bytes.
 // and append() takes a variadic as it's second parameter, so these "unpacked" individual bytes 
 // can now be appended to the first paramater []byte, which is a slice of bytes.  yes, it works!
 // Let's call this working version: V1
 b.Write(append(newData, []byte(",")...))
 fmt.Println(b.String())
 
 // BTW, we could just make two writes to the buffer.
 // this is V2 and it works too, but why is it worse than V2 for our use-case of
 // implementing the io.Writer?
 b.Reset()
 b.Write(newData)
 b.Write([]byte(","))
 fmt.Println(b.String())
 
}



No comments:

Post a Comment