Go 中的切片分块

Question

我有一个包含大约 210 万个日志字符串的切片，我想创建一个切片，其中的字符串尽可能均匀分布。

这是我到目前为止所拥有的：

// logs is a slice with ~2.1 million strings in it.

var divided = make([][]string, 0)

NumCPU := runtime.NumCPU()

ChunkSize := len(logs) / NumCPU

for i := 0; i < NumCPU; i++ {

    temp := make([]string, 0)

    idx := i * ChunkSize

    end := i * ChunkSize + ChunkSize

    for x := range logs[idx:end] {

        temp = append(temp, logs[x])

    }

    if i == NumCPU {

        for x := range logs[idx:] {

            temp = append(temp, logs[x])

        }

    }

    divided = append(divided, temp)

}

该idx := i * ChunkSize会给我目前的“块启动”为logs索引，end := i * ChunkSize + ChunkSize会给我“块结束”，或该块的范围的结束。我找不到任何关于如何在 Go 中分块/拆分切片或在有限范围内迭代的文档或示例，所以这就是我想出的。但是，它只会多次复制第一个块，因此它不起作用。

我如何（尽可能均匀地）在 Go 中分块？

Answer 1

您不需要制作新切片，只需将切片附加logs到divided切片即可。http://play.golang.org/p/vyihJZlDVyvar divided [][]stringchunkSize := (len(logs) + numCPU - 1) / numCPUfor i := 0; i < len(logs); i += chunkSize {&nbsp; &nbsp; end := i + chunkSize&nbsp; &nbsp; if end > len(logs) {&nbsp; &nbsp; &nbsp; &nbsp; end = len(logs)&nbsp; &nbsp; }&nbsp; &nbsp; divided = append(divided, logs[i:end])}fmt.Printf("%#v\n", divided)

Answer 2

另一种变体。它的工作速度比JimB提出的快 2.5 倍。测试和基准测试在这里。https://play.golang.org/p/WoXHqGjozMIfunc chunks(xs []string, chunkSize int) [][]string {&nbsp; &nbsp; if len(xs) == 0 {&nbsp; &nbsp; &nbsp; &nbsp; return nil&nbsp; &nbsp; }&nbsp; &nbsp; divided := make([][]string, (len(xs)+chunkSize-1)/chunkSize)&nbsp; &nbsp; prev := 0&nbsp; &nbsp; i := 0&nbsp; &nbsp; till := len(xs) - chunkSize&nbsp; &nbsp; for prev < till {&nbsp; &nbsp; &nbsp; &nbsp; next := prev + chunkSize&nbsp; &nbsp; &nbsp; &nbsp; divided[i] = xs[prev:next]&nbsp; &nbsp; &nbsp; &nbsp; prev = next&nbsp; &nbsp; &nbsp; &nbsp; i++&nbsp; &nbsp; }&nbsp; &nbsp; divided[i] = xs[prev:]&nbsp; &nbsp; return divided}

Answer 3

对任何 []T 使用反射https://github.com/kirito41dd/xslicepackage mainimport (&nbsp; &nbsp; "fmt"&nbsp; &nbsp; "github.com/kirito41dd/xslice")func main() {&nbsp; &nbsp; s := []int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}&nbsp; &nbsp; i := xslice.SplitToChunks(s, 3)&nbsp; &nbsp; ss := i.([][]int)&nbsp; &nbsp; fmt.Println(ss) // [[0 1 2] [3 4 5] [6 7 8] [9]]}https://github.com/kirito41dd/xslice/blob/e50d91fa75241a3a03d262ad51c8e4cb2ea4b995/split.go#L12func SplitToChunks(slice interface{}, chunkSize int) interface{} {&nbsp; &nbsp; sliceType := reflect.TypeOf(slice)&nbsp; &nbsp; sliceVal := reflect.ValueOf(slice)&nbsp; &nbsp; length := sliceVal.Len()&nbsp; &nbsp; if sliceType.Kind() != reflect.Slice {&nbsp; &nbsp; &nbsp; &nbsp; panic("parameter must be []T")&nbsp; &nbsp; }&nbsp; &nbsp; n := 0&nbsp; &nbsp; if length%chunkSize > 0 {&nbsp; &nbsp; &nbsp; &nbsp; n = 1&nbsp; &nbsp; }&nbsp; &nbsp; SST := reflect.MakeSlice(reflect.SliceOf(sliceType), 0, length/chunkSize+n)&nbsp; &nbsp; st, ed := 0, 0&nbsp; &nbsp; for st < length {&nbsp; &nbsp; &nbsp; &nbsp; ed = st + chunkSize&nbsp; &nbsp; &nbsp; &nbsp; if ed > length {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ed = length&nbsp; &nbsp; &nbsp; &nbsp; }&nbsp; &nbsp; &nbsp; &nbsp; SST = reflect.Append(SST, sliceVal.Slice(st, ed))&nbsp; &nbsp; &nbsp; &nbsp; st = ed&nbsp; &nbsp; }&nbsp; &nbsp; return SST.Interface()}