README
¶
Cbyte
Provides low-level API to manipulate with byte arrays in C memory.
The main goal is reducing allocations and consecutive reduce GC pressure.
Usage
package main
import (
"fmt"
"github.com/koykov/cbyte"
)
func main() {
var words = [][]byte{
[]byte("foo"),
[]byte("bar"),
[]byte("string"),
[]byte("of"),
[]byte("example"),
}
// Next line is equivalent of make([]byte, 0, 30), but it produces a byte slice
// without using of runtime.mallocgc().
// Therefore GC doesn't know anything about this slice and just ignore it during
// both GC mark and GC termination phases.
p := cbyte.InitBytes(0, 30)
for _, w := range words {
p = append(p, w...)
}
fmt.Println(p)
// This is the main inconvenience: need to resolve cbyte slices and strings
// manually after using.
cbyte.ReleaseBytes(p)
}
How it works
As we know slices and strings in Go bases on the underlying arrays of corresponding types. Each modification of these arrays triggers new allocations that included:
- allocation of new array with required size;
- copy data from the old array.
After that old array (and corresponding slice or strings) become a garbage and need to be collected during next GC cycle.
My idea is to move all low-level operations with underlying arrays to C memory, outside of GC eyes. Each allocation, grow and free of the array triggers CGO calls, that performs low-level manipulation with memory (see cbyte.c). All other operations like append(), copy(), ... works the same as on regular slices and strings.
It is a moot approach but it works. Just need to be a bit more careful.
Documentation
¶
Index ¶
- func Bytes(h reflect.SliceHeader) []byte
- func Grow(addr uint64, capOld, cap int) uint64
- func Grow64(addr uint64, capOld, cap uint64) uint64
- func GrowHeader(h reflect.SliceHeader) uint64
- func GrowHeader64(h SliceHeader64) uint64
- func Header(p []byte) reflect.SliceHeader
- func HeaderStr(p string) reflect.SliceHeader
- func HeaderString(p string) reflect.SliceHeader
- func Init(cap int) uint64
- func Init64(cap uint64) uint64
- func InitBytes(len, cap int) []byte
- func InitHeader(len, cap int) reflect.SliceHeader
- func InitStr(len int) string
- func InitString(len int) string
- func Memcpy(addr, offset uint64, data []byte) (n int)
- func RegisterMetricsHandler(handler MetricsWriter)
- func Release(addr uint64)
- func ReleaseBytes(p []byte)
- func ReleaseHeader(h reflect.SliceHeader)
- func ReleaseHeader64(h SliceHeader64)
- func ReleaseStr(p string)
- func ReleaseString(p string)
- func Str(h reflect.SliceHeader) string
- func String(h reflect.SliceHeader) string
- type DummyMetrics
- type MetricsWriter
- type SliceHeader64
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Grow ¶
Grow increases capacity of the byte array.
All necessary copying/free will perform implicitly, don't worry about this.
func GrowHeader ¶
func GrowHeader(h reflect.SliceHeader) uint64
GrowHeader increases capacity of the byte array using SliceHeader.
func GrowHeader64 ¶
func GrowHeader64(h SliceHeader64) uint64
GrowHeader64 increases capacity using SliceHeader64.
func HeaderStr ¶
func HeaderStr(p string) reflect.SliceHeader
HeaderStr decomposes string to SliceHeader. DEPRECATED: use HeaderString() instead.
func HeaderString ¶ added in v1.0.5
func HeaderString(p string) reflect.SliceHeader
HeaderString decomposes string to SliceHeader.
func InitHeader ¶
func InitHeader(len, cap int) reflect.SliceHeader
InitHeader makes slice header of byte array.
func InitStr ¶
InitStr makes a string with underlying cbyte pointer. DEPRECATED: use InitString() instead.
func InitString ¶ added in v1.0.5
InitString makes a string with underlying cbyte pointer.
func RegisterMetricsHandler ¶
func RegisterMetricsHandler(handler MetricsWriter)
RegisterMetricsHandler register new metrics handler.
func ReleaseBytes ¶
func ReleaseBytes(p []byte)
ReleaseBytes free underlying cbyte slice.
Caution! Don't try to release non-cbyte slices.
func ReleaseHeader ¶
func ReleaseHeader(h reflect.SliceHeader)
ReleaseHeader free byte array using SliceHeader.
func ReleaseHeader64 ¶
func ReleaseHeader64(h SliceHeader64)
ReleaseHeader64 releases byte array using SliceHeader64.
func ReleaseStr ¶
func ReleaseStr(p string)
ReleaseStr releases underlying cbyte pointer of string.
Caution! Don't try to release non-cbyte strings. DEPRECATED: use ReleaseString() instead.
func ReleaseString ¶ added in v1.0.5
func ReleaseString(p string)
ReleaseString releases underlying cbyte pointer of string.
Caution! Don't try to release non-cbyte strings.
func Str ¶
func Str(h reflect.SliceHeader) string
Str composes string from SliceHeader. DEPRECATED: use String() instead.
func String ¶ added in v1.0.5
func String(h reflect.SliceHeader) string
String composes string from SliceHeader.
Types ¶
type DummyMetrics ¶
type DummyMetrics struct{}
DummyMetrics writer. Used by default and does nothing.
func (DummyMetrics) Alloc ¶
func (m DummyMetrics) Alloc(_ uint64)
func (DummyMetrics) Free ¶
func (m DummyMetrics) Free(_ uint64)
func (DummyMetrics) Grow ¶
func (m DummyMetrics) Grow(_, _ uint64)
type MetricsWriter ¶
type MetricsWriter interface {
// Alloc register size of new allocated space.
Alloc(size uint64)
// Grow registers growing of cbyte object from sizeOld to sizeNew.
Grow(sizeOld, sizeNew uint64)
// Free registers freeing of cbyte object with given size.
Free(size uint64)
}
MetricsWriter interface.
type SliceHeader64 ¶
SliceHeader64 represents runtime representation of a big byte slice. Allows to exceed MAXINT limit for length and capacity.
func InitHeader64 ¶
func InitHeader64(len, cap uint64) SliceHeader64
InitHeader64 makes return SliceHeader64 of big byte array.
Source Files
Directories