Can we write a general array / slice deduplication in go?

Question

Is there a way to write a general array / slice deduplication in go, for []int we can have something like (from http://rosettacode.org/wiki/Remove_duplicate_elements#Go ):

 func uniq(list []int) []int { unique_set := make(map[int] bool, len(list)) for _, x := range list { unique_set[x] = true } result := make([]int, len(unique_set)) i := 0 for x := range unique_set { result[i] = x i++ } return result } 

But is there a way to expand it to support any array? with signatures like:

 func deduplicate(a []interface{}) []interface{} 

I know that you can write this function with this signature, but then you cannot use it on []int , you need to create []interface{} to put everything from []int into it, pass it to the function, then it will return and put in the []interface{} and go through this new array and put everything in the new []int .

My question is: is there a better way to do this?

Answer 1

While VonC's answer is probably closest to what you really want, the only real way to do it in native Go without gen is to define an interface

 type IDList interface { // Returns the id of the element at i ID(i int) int // Returns the element // with the given id GetByID(id int) interface{} Len() int // Adds the element to the list Insert(interface{}) } // Puts the deduplicated list in dst func Deduplicate(dst, list IDList) { intList := make([]int, list.Len()) for i := range intList { intList[i] = list.ID(i) } uniques := uniq(intList) for _,el := range uniques { dst.Insert(list.GetByID(el)) } } 

Where uniq is a function of your OP.

This is just one possible example, and probably much better, but in the general case, matching each item with a unique identifier "== able" and either creating a new list or filtering based on identifier deduplication is probably the most intuitive way.

An alternative solution is to take []IDer , where the IDer interface is equal to ID() int . However, this means that custom code must create a list of [] IDer and copy all the elements into this list, which is a bit ugly. It is cleaner for the user to wrap the list as a list of identifiers rather than copy, but this is a similar amount of work anyway.

Answer 2

The only way I've seen implemented in Go is with the clipperhouse / gen project,

gen - An attempt to bring some common features like Go, with some inspiration from the linux libraries and JavaScripts from C #

See this test :

 // Distinct returns a new Thing1s slice whose elements are unique. See: http://clipperhouse.imtqy.com/gen/#Distinct func (rcv Thing1s) Distinct() (result Thing1s) { appended := make(map[Thing1]bool) for _, v := range rcv { if !appended[v] { result = append(result, v) appended[v] = true } } return result } 

But , as described in clipperhouse.imtqy.com/gen/ :

gen generates code for your types at design time using the command line.

gen is not an import; the generated source becomes part of your project and does not accept external dependencies.

Answer 3

You can do something close to this through an interface. Define an interface, say, “DeDupable”, requiring func, say UniqId () [] byte, which can then be used to remove duplicates. and your uniq func will use [] DeDupable and work on it