Documentation
¶
Index ¶
- Constants
- Variables
- func MergeFlds(parsedRec *fastjson.Value, flds []FldFormat, separator string) (mergedVal string, err error)
- func PlainString(in string) string
- func Trace(msg string)
- type BktRequest
- type BobbErr
- type CopyDBRequest
- type CsvExport
- type DeleteRequest
- type ExportRequest
- type FindCondition
- type FindGroup
- type FldFormat
- type GetAllKeysRequest
- type GetAllRequest
- type GetOneRequest
- type GetRequest
- type GetValuesRequest
- type GlobalVal
- type IndexKeyVal
- type IndexRequest
- type IndexSetting
- type IndexSettingRequest
- type Indexr
- type Join
- type PutIndexRequest
- type PutParm
- type PutRequest
- type QryRequest
- type ReadLoop
- type RecValues
- type Request
- type Response
- type SearchKeysRequest
- type SortKey
- type SortRec
- type VerifyIndexRequest
Constants ¶
const ( OpBkt = "bkt" OpGet = "get" OpGetOne = "getone" OpGetAll = "getall" OpGetAllKeys = "getallkeys" OpQry = "qry" OpPut = "put" OpPutIndex = "putindex" OpDelete = "delete" OpVerifyIndex = "verifyindex" OpIndexSetting = "indexsetting" OpIndexRequest = "indexrequest" OpExport = "export" OpClose = "close" OpCopyDB = "copydb" )
Request Operations
const ( StatusOk = "ok" StatusFail = "fail" StatusWarning = "warning" )
Response Status Values
const ( SortAscStr = "ascstr" SortDescStr = "descstr" SortAscInt = "ascint" SortDescInt = "descint" )
SortKey Dir Codes
const ( FindContains = "contains" // str - substring match FindContainsWord = "containsword" // str - whole word match FindMatches = "matches" // str - exact match FindStartsWith = "startswith" // str - prefix match FindEndsWith = "endswith" // str - suffix match FindBefore = "before" // str - less than compare value FindAfter = "after" // str - greater than compare value FindInStrList = "instrlist" // str - in list FindLessThan = "lessthan" // int FindGreaterThan = "greaterthan" // int FindEquals = "equals" // int FindInIntList = "inintlist" // int FindExists = "exists" // any type FindIsNull = "isnull" // any type FindNot = true // used to set FindCondition.Not field )
FindCondition Op Codes
const ( ErrNotFound = "notfound" // specified key not found in bkt ErrIndexRef = "indexref" // index value not key in bkt ErrParseRec = "parserec" // error parsing record ErrFldNotFound = "fldnotfound" // fld not found in record ErrFldIsNull = "fldisnull" // value of fld is null ErrFldType = "fldtype" // fld type in bkt rec does not match req fld type ErrJoinBkt = "joinbkt" // join bkt not found ErrJoinFld = "joinfld" // join fld invalid ErrJoinKey = "joinkey" // join key not found in join bkt ErrJoinParse = "joinparse" // error parsing join record ErrJoinFromFld = "joinfromfld" // join from fld invalid // Verify Index Errors ErrInvalidIndexValue = "invalidindexvalue" // ErrDuplicateIndexValue = "duplicateindexvalue" // ErrDataKeyNotIndexed = "datakeynotindexed" // )
Bobb Error Codes, Used for BobbError.ErrCode value
const ( DefaultAlways = "always" // on not found or null return zero value DefaultNever = "never" // return error if notfound or null DefaultIsNull = "isnull" // if null, return zero value, not found is error DefaultNotFound = "notfound" // if not found, return zero value, null is error )
UseDefault Codes, controls value returned when record field not found or is null Default (zero value) is "" for string and 0 for int
const ( StrLowerCase = "lowercase" // value converted to lowercase StrPlain = "plain" // value converted to lowercase + non alphanumeric removed StrAsIs = "asis" // value used asis, no changes )
const ( BktCreate = "create" // create bucket BktDelete = "delete" // delete bucket BktNextSeq = "nextseq" // get next sequence number(s) BktList = "list" // get list of all buckets in db BktCount = "count" // get count of keys in a bucket )
values for BktRequest Operation
const ( FldTypeStr = "string" FldTypeInt = "int" )
const ( IndexingNormal = "normal" // adds and updates to index bkts (most processing) IndexingOff = "off" // no adds or updates to index bkts IndexingNoUpdate = "noupdate" // no updates to index bkts, only adds (no checks for index already existing for data key) )
PutRequest IndexingOption Codes (IndexingNormal default)
const CreateIfNotExists = true
const IndexSettingsBkt = "index_settings"
const (
LogPut = true // used for PutParm.LogPut field, indicates record should be written to put log bkt
)
misc codes
const OpGetValues = "getvalues"
const OpSearchKeys = "searchkeys"
Variables ¶
var AllDefaultCodes = []string{DefaultAlways, DefaultNever, DefaultIsNull, DefaultNotFound}
var AllFindOps = slices.Concat(StrFindOps, IntFindOps, []string{FindExists, FindIsNull})
var AllIndexingOptions = []string{IndexingNormal, IndexingOff, IndexingNoUpdate}
var AllSortCodes = slices.Concat(StrSortCodes, IntSortCodes)
var AllStrOptions = []string{StrLowerCase, StrPlain, StrAsIs}
var AscSortCodes = []string{SortAscInt, SortAscStr}
var DefaultKeyFld string // set at startup by bobb_server.go, used in Put requests that don't specify KeyField
var DescSortCodes = []string{SortDescInt, SortDescStr}
var ErrBadInputData error = errors.New("data error - bad json or no key") // used by put funcs when input data has problems
var InitialRespRecsSize int // from bobb_settings.json, response.Recs slice initial allocation for this size
var IntFindOps = []string{FindLessThan, FindGreaterThan, FindEquals, FindInIntList}
var IntSortCodes = []string{SortAscInt, SortDescInt}
var KeySuffixWidth int // from bobb_settings.json, set at startup by bobb_server.go, used in Put requests with AddKeySuffix true
var MaxErrs int // from bobb_settings.json, set at startup by bobb_server.go
var StrFindOps = []string{FindContains, FindContainsWord, FindMatches, FindStartsWith, FindEndsWith, FindBefore, FindAfter, FindInStrList}
var StrSortCodes = []string{SortAscStr, SortDescStr}
Functions ¶
func MergeFlds ¶
func MergeFlds(parsedRec *fastjson.Value, flds []FldFormat, separator string) (mergedVal string, err error)
MergeFlds is typically used to create index keys composed of multiple flds merged together. Type FldFormat defined in types.go (FldName, FldType, Length, StrOption, UseDefault). Separator placed between each value. Each fld value is truncated or padded if needed to set length. UseDefault controls what happens when fld is not found or null in parsedRec. Ex. DefaultNull would return 0 for FldTypeInt, or "" for FldTypeStr if fld value is null. DefaultNever would return error if fld value is null or fld not found.
func PlainString ¶
PlainString returns lower case version of input string with non alphanumeric chars removed. Ex. Hip-Hop > hiphop Used by MergeFlds below.
Types ¶
type BktRequest ¶
type BktRequest struct {
BktName string
Operation string // "create", "delete", "nextseq", "list", "count"
NextSeqCount int // used with nextseq op to specify how many (max 100)
}
BktRequest performs bucket requests: "create", "delete", "nextseq", "list", "count". For "nextseq" operations: if NextSeqCount = 0, 1 value is returned in NextSeq. Note - a maximum of 100 seq #'s are returned per request. List operation returns names of all bkts in resp.Recs. BktName fld not used. See shortcut func GetBktList() in client/util.go. Count operation returns number of keys in specified bkt. See shortcut func GetRecCount() in client/util.go.
func (BktRequest) IsUpdtReq ¶
func (req BktRequest) IsUpdtReq() bool
type CopyDBRequest ¶
type CopyDBRequest struct {
FilePath string `json:"filePath"`
}
CopyDB copies the open db to another file. Does not block other operations.
func (CopyDBRequest) IsUpdtReq ¶
func (req CopyDBRequest) IsUpdtReq() bool
type DeleteRequest ¶
DeleteRequest is used to delete specific records by Key. Keys not found are ignored.
func (DeleteRequest) IsUpdtReq ¶
func (req DeleteRequest) IsUpdtReq() bool
type ExportRequest ¶
type ExportRequest struct {
BktName string
StartKey string // if not "", keys >= this value
EndKey string // if not "", keys <= this value
Limit int // max # recs to write
FilePath string // where export file is written
}
Export writes bkt records to a file as formatted json.
func (ExportRequest) IsUpdtReq ¶
func (req ExportRequest) IsUpdtReq() bool
type FindCondition ¶
type FindCondition struct {
Fld string // field containing compare value
Op string // defines match operation and value type
ValStr string // for string ops, this value also converted based on StrOption
ValInt int // for int Ops
StrList []string // used by op FindInStrList
IntList []int // used by op FindInIntList
Not bool // exclude records that meet condition
UseDefault string // controls what default value is used, see Default* codes in codes.go
StrOption string // controls string conversion, see Str* codes in codes.go, default StrLowerCase
}
FindCondition is used by QryRequest to define select criteria. Each record's Fld value is compared to FindCondition value. See codes.go for Find* op code constants.
type FindGroup ¶
type FindGroup []FindCondition // QryRequest can have multiple FindGroups that are ORed together
type FldFormat ¶
type FldFormat struct {
FldName string // name of fld in record
FldType string // FldTypeStr or FldTypeInt ("string" or "int")
Length int // output length of value
UseDefault string // controls value used when fld not found or null in data rec, use constant from codes.go: DefaultAlways, DefaultNever, DefaultIsNull, DefaultNotFound
StrOption string // for string flds, use Str* code (see codes.go) to control conversion, ex. StrLowerCase
}
FldFormat is used by MergeFlds in rec.go, typically for creating index keys. Strings - padded to right with spaces or truncated as needed. Ints - leading zeros added as needed.
type GetAllKeysRequest ¶
type GetAllKeysRequest struct {
BktName string
StartKey string // if not "", keys >= this value
EndKey string // if not "", keys <= this value
Limit int // max # recs to return
}
GetKeys returns keys, not values, from specified bucket. Keys are returned in the Response.Recs as json.Marshaled string. Use Start/End keys to specify range to be included. If StartKey == EndKey, key prefix must match StartKey.
func (GetAllKeysRequest) IsUpdtReq ¶
func (req GetAllKeysRequest) IsUpdtReq() bool
type GetAllRequest ¶
type GetAllRequest struct {
BktName string
IndexBkt string // name of bkt used as index
StartKey string // if not "", keys >= this value
EndKey string // if not "", keys <= this value
Limit int // max # recs to return
ErrLimit int // run stops when ErrLimit exceeded, default 0, settings.MaxErrs limit if -1
}
GetAllRequest returns all records in bucket or records in range between Start/End keys. Records are returned in key order. If StartKey == EndKey, rec key prefix must match StartKey. If StartKey = "", reads from beginning. If EndKey = "" reads to end. If end of bkt not reached, response.NextKey will be next key in order.
func (GetAllRequest) IsUpdtReq ¶
func (req GetAllRequest) IsUpdtReq() bool
type GetOneRequest ¶
GetOneRequest is used to get a specific record by Key.
func (GetOneRequest) IsUpdtReq ¶
func (req GetOneRequest) IsUpdtReq() bool
type GetRequest ¶
type GetRequest struct {
BktName string
Keys []string // keys of records to be returned
ErrLimit int // run stops when ErrLimit exceeded
}
GetRequest is used to get specific records by key.
func (GetRequest) IsUpdtReq ¶
func (req GetRequest) IsUpdtReq() bool
type GetValuesRequest ¶
type GetValuesRequest struct {
BktName string
Keys []string // keys of records to be returned
Fields []string // field values to return
}
GetValues returns specific values rather than entire record. Default field type is string. To get other types add "|type" to field name. Example "count|int". All return values converted to string. If value cannot be extracted from record, return value is empty string. Response.Recs loaded with slice of json marshalled RecValues (see type above) Valid type values: string, int, float64, bool (defaults to string)
func (GetValuesRequest) IsUpdtReq ¶
func (req GetValuesRequest) IsUpdtReq() bool
type IndexKeyVal ¶
type IndexKeyVal struct {
Key string
Val string
OldKey string // used when index rec already exists for data key
}
IndexKeyVal type is used by PutIndexRequest. Key is typically created from value(s) in data record (must be made unique). Val is key of record in data bkt. If OldKey not empty, it will be deleted. No problem if it does not exist. MergeFlds func in rec.go can be used to merge multiple flds together to form key.
type IndexRequest ¶
type IndexRequest struct {
DataBkt string // name of data bkt, DataKeys refer to this bkt
IndexBkt string // name of index bkt, where index entries will be written
MergeFlds []FldFormat // defines index composition
FldSeparator string // separator used in merged field values
KeySuffixWidth int // using IndexBkt nextSeq# add numeric suffix to index key, 0 means use KeySuffixWidth from bobb_setting.json, -1 no suffix
DataKeys []string // index specific data records
StartKey string // index records in range from StartKey
EndKey string // index records in range to EndKey
IndexAll bool // index all records in DataBkt
SkipOnErrLimit int // if errors exceed this limit, fail request with rollback
}
IndexRequest is used to add index records for specific data keys, keys in a range, or all keys in a data bkt. Only one indexing mode can be used per request:
- IndexAll: true — index all records in DataBkt
- StartKey/EndKey — index records within key range
- DataKeys — index specific records by key
Errors are collected in resp.Errs until SkipOnErrLimit is exceeded. Warning - if there is potential for the result of MergeFlds to not be unique, a KeySuffix is required
func (IndexRequest) IsUpdtReq ¶
func (req IndexRequest) IsUpdtReq() bool
type IndexSetting ¶
type IndexSetting struct {
DataBkt string // name of data bkt, ex. "inquiry"
IndexBkt string // name of index bkt, must begin with value of DataBkt and end with "_index", ex. "inquiry_timestamp_index"
KeyFlds []FldFormat // defines how index key is constructed, if empty, index key is just IndexBkt NextSequence#
FldSeparator string // optional separator used in merged field values, ex. "|" > "critical |00033|temp high "
KeySuffixWidth int // using IndexBkt nextSeq# add numeric suffix to index key, 0 means use KeySuffixWidth from bobb_setting.json, -1 no suffix
SkipOnErr bool // if true, if error creating/updating index entry for a data rec, skip and do not fail entire PutRequest
}
IndexSetting defines how index entries are constructed for a data bkt. They are loaded into the "index_settings" bkt. PutRequests use these settings to create/update index entries automatically when data records are added/updated in the data bkt.
KeySuffixWidth is used to pad the index key suffix to fixed width with leading zeros. This ensures proper sorting of index keys. Example - if KeySuffixWidth is 6, index keys will end with suffixes like "000001", "000002", ..., "000010", etc.
type IndexSettingRequest ¶
type IndexSettingRequest struct {
IndexSettings []IndexSetting
}
IndexSettingRequest loads IndexSettings into the "index_settings" bkt. Key is value of IndexSetting.IndexBkt. Val is json.Marshalled instance of IndexSetting.
func (IndexSettingRequest) IsUpdtReq ¶
func (req IndexSettingRequest) IsUpdtReq() bool
type Indexr ¶
type Indexr struct {
IndexBkt *bolt.Bucket // pointer to index bkt where index entries are stored
IndexBktName string // name of index bkt, used for error msgs
IndexInvertedBkt *bolt.Bucket // maps data key to index key, so easy to find old index entry for data key
KeyFlds []FldFormat // defines how index key, is constructed
FldSeparator string // separator used in merged field values
KeySuffixFormat string // using IndexBkt nextSeq#, formatted with leading zeros
SkipOnErr bool // if true, on error skip writing index entry and don't fail PutRequest
}
Indexr is used to perform indexing (add/change index rec) for a data bkt based on an IndexSetting (see requests_index.go). Indexrs are created and run by PutRequest, see requests_put.go.
type Join ¶
type Join struct {
JoinBkt string // name of related bkt where value is pulled from
JoinFld string // fld in primary rec containing key value of join rec
FromFld string // fld in join rec where value comes from
ToFld string // fld in primary rec where value is loaded
UseDefault bool // if join problem, use default value, no error
}
type PutIndexRequest ¶
type PutIndexRequest struct {
BktName string
Indexes []IndexKeyVal // slice of index key/val/oldkey structs
}
PutIndex is used to add or replace index records. Bolt Put rules apply: if key does not exist, rec is added, else rec is replaced. Key is field value(s) from primary bkt (made unique). Val is key of record in primary bkt. WARNING - if data rec already has index rec, changing index key will cause multiple records for same data rec. Use OldKey to delete existing index rec.
func (PutIndexRequest) IsUpdtReq ¶
func (req PutIndexRequest) IsUpdtReq() bool
type PutParm ¶
type PutParm struct {
BktName string // data bkt where recs will be put, created if not exists
KeyField string // fld in recs containing key value, default is defaultKeyFld from bobb_settings.json
Recs [][]byte // typically json marshaled value of records
RequiredFlds []string // optional, fld names that must be included in recs
AddKeySuffix bool // if true, add bkt NextSeq# to end of key
IndexingOption string // see Indexing* codes in codes.go, IndexingNormal is default
LogPut bool // if true, write record to bktname_putlog bkt. Key is dataKey|timestamp. Value is Rec. Provides point in time values.
}
PutParm(s) used by PutRequest to specify parameters for each put operation.
type PutRequest ¶
type PutRequest struct {
PutParms []PutParm
}
PutRequest is used to add or replace records. Multiple PutParms can be included in a single request, allows for multiple bkts to be updated in a single transaction.
func (PutRequest) IsUpdtReq ¶
func (req PutRequest) IsUpdtReq() bool
type QryRequest ¶
type QryRequest struct {
BktName string // primary data bkt
IndexBkt string // optional index bkt name, start/end keys use index
Criteria []FindGroup // if a record meets all conditions in any FindGroup, it is included in results
SortKeys []SortKey // defines sort order, if omitted ressults returned in key order
StartKey string // begin range, 1st key >=
EndKey string // end range, last key <=
Limit int // limits results before sort step
Top int // limits results after sort step
ErrLimit int // run stops when ErrLimit exceeded, default 0, settings.MaxErrs limit if -1
JoinsBeforeFind []Join // joined values can be used in find step (adds processing time)
JoinsAfterFind []Join // joined values can be used for sort step but not find step
CountOnly bool // if true, Response.Recs is nil, count in Response.GetCnt
}
QryRequest is used to filter and sort recs from a bkt. Start/End keys define range of keys to read. If StartKey == EndKey, key prefix must match StartKey.
func (QryRequest) IsUpdtReq ¶
func (req QryRequest) IsUpdtReq() bool
type ReadLoop ¶
type ReadLoop struct {
Bkt *bolt.Bucket // data bkt
Index *bolt.Bucket // index bkt
Csr *bolt.Cursor // if index set, index csr, else data bkt csr
StartKey string // begin loop with 1st key >= StartKey
EndKey string // end loop with 1st key > EndKey
MatchPrefix bool // if StartKey == EndKey, rec key prefix must match StartKey
UsingIndex bool // indicates if index is being used
NextKey []byte // used for resp.NextKey when range-end or limit hit
Limit int // results limit
Count int // Count equal Limit triggers loop end, Count updated by caller
}
ReadLoop type provides functionality for reading bucket records sequentially. Optional Index, StartKey, EndKey
func NewReadLoop ¶
Create and return new instance of ReadLoop. Parm bkt is pointer to data bucket. Parm index is pointer to index bucket. If nil, no index.
func (*ReadLoop) Next ¶
Next returns next key/value pair or nil/nil if loop ended. If UsingIndex, key is index key. Value is always from data bkt. If k is outside of range, loop.NextKey loaded with k.
type Response ¶
type Response struct {
Status string // constants in codes.go (StatusOk, StatusWarning, StatusFail)
Msg string // if status is not Ok, Msg will indicate reason
Recs [][]byte // for request responses with potentially more than 1 record
Rec []byte // for requests that only return 1 record
PutCnt int // number of records either added or replaced by Put operation
PutKeys map[int][]string // keys used in PutRequest (includes appended suffix if used), map key is PutParms index
GetCnt int // used for other non Put counts
NextSeq []int // returned by Bkt request with Operation = "nextseq"
NextKey string // next key in bkt after last one returned in Recs
Errs []BobbErr // errs occuring until req.ErrLimit hit
}
Response type is returned by all db requests. Individual recs must be json.Unmarshaled into appropriate type by receiver.
NOTE - PutKeys are separated by PutParm, PutKeys[0] contains keys used for PutRequest.PutParms[0].Recs.
type SearchKeysRequest ¶
type SearchKeysRequest struct {
BktName string
SearchValue string
StartKey string
EndKey string
Limit int
}
SearchKeys returns records where key contains search value. If bkt is an index bkt, the returned value is the key of the indexed data record.
func (SearchKeysRequest) IsUpdtReq ¶
func (req SearchKeysRequest) IsUpdtReq() bool
type SortKey ¶
type SortKey struct {
Fld string // name of field
Dir string // direction (asc/desc) and field type (str/int)
UseDefault string // controls what value is used when fld NotFound or IsNull, see codes.go
}
SortKey is used by QryRequest to sort results. Only fields of type string or int are currently supported. String values are converted to "plain" string (lowercase, alphanumeric).
type SortRec ¶
type SortRec struct {
SortOn []string // Values extracted from record using SortKeys
Value []byte // record value
}
SortRec is used when QryRequest has SortKeys
type VerifyIndexRequest ¶
type VerifyIndexRequest struct {
DataBkt string // data bkt
IndexBkt string //
AllDataIndexed bool //if true, verify all data keys have entry in index
ErrLimit int // limit of BobbErr's returned in Response.Errs
}
VerifyIndexRequest verifies index records are valid. Check index values are unique and refer to an existing data key. If AllDataIndexed true, verify all data keys have entry in index.
func (VerifyIndexRequest) IsUpdtReq ¶
func (req VerifyIndexRequest) IsUpdtReq() bool
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
demo.go exercises all bobb request types and verifies results are correct.
|
demo.go exercises all bobb request types and verifies results are correct. |