datastore

package module
v1.3.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 26, 2026 License: MIT Imports: 30 Imported by: 19

README

💾  go-datastore

Data layer using GORM for accessing models via a Database (MySQL, Postgres, SQLite, MongoDB).


Release Go Version License


CI / CD    Build Last Commit      Quality    Go Report Coverage
Security    Scorecard Security      Community    Contributors Bitcoin


Project Navigation
🚀 Installation 🧪 Examples & Tests 📚 Documentation
🤝 Contributing 🛠️ Code Standards ⚡ Benchmarks
🤖 AI Usage ⚖️ License 👥 Maintainers

Installation

go-datastore requires a supported release of Go.

go get github.com/mrz1836/go-datastore

Documentation

View the generated documentation

GoDoc

Database model assumptions:

  • id If a unique ID is needed, the datastore expects id and uses _id internally for Mongo
  • metadata is an optional field for storing key->value JSON data

Custom array and object fields:

  • WithCustomFields(arrayFields, objectFields ) Use this method to add custom array or object fields (IE: metadata)

Custom methods for Mongo:

  • WithCustomMongoConditionProcessor() Use this method to add custom condition processing for custom object fields
  • WithCustomMongoIndexer() Use this method to add custom mongo indices

Development Setup (Getting Started)

Install MAGE-X build tool for development:

# Install MAGE-X for development and building
go install github.com/mrz1836/mage-x/cmd/magex@latest
magex update:install
Library Deployment

This project uses goreleaser for streamlined binary and library deployment to GitHub. To get started, install it via:

brew install goreleaser

The release process is defined in the .goreleaser.yml configuration file.

Then create and push a new Git tag using:

magex version:bump bump=patch push=true branch=master

This process ensures consistent, repeatable releases with properly versioned artifacts and citation metadata.

Build Commands

View all build commands

magex help
GitHub Workflows

All workflows are driven by modular configuration in .github/env/ — no YAML editing required.

View all workflows and the control center →

Updating Dependencies

To update all dependencies (Go modules, linters, and related tools), run:

magex deps:update

This command ensures all dependencies are brought up to date in a single step, including Go modules and any managed tools. It is the recommended way to keep your development environment and CI in sync with the latest versions.


Examples & Tests

All unit tests run via GitHub Actions and use Go version 1.25.x. View the configuration file.

Run all tests (fast):

magex test

Run all tests with race detector (slower):

magex test:race

Benchmarks

Run the Go benchmarks:

magex bench

Code Standards

Read more about this Go project's code standards.


🤖 AI Usage & Assistant Guidelines

Read the AI Usage & Assistant Guidelines for details on how AI is used in this project and how to interact with AI assistants.


👥 Maintainers

MrZ
MrZ

Contributing

View the contributing guidelines and please follow the code of conduct.

How can I help?

All kinds of contributions are welcome 🙌! The most basic way to show your support is to star 🌟 the project, or to raise issues 💬. You can also support this project by becoming a sponsor on GitHub 👏 or by making a bitcoin donation to ensure this journey continues indefinitely! 🚀

Stars


License

License

Documentation

Overview

Package datastore is the database service abstraction layer

Index

Constants

View Source 
const (

	// SortDesc will sort descending
	SortDesc = "desc"

	// SortAsc will sort ascending
	SortAsc = "asc"
)

Defaults for library functionality

View Source 
const (
	Postgres = "postgres"
	JSON     = "JSON"
	JSONB    = "JSONB"
)

index creation constants

Variables

View Source 
var DateFields = []string{dateCreatedAt, dateUpdatedAt, dateModifiedAt} //nolint:gochecknoglobals // exported package-level constant for date fields

DateFields are standard known date fields

View Source 
var ErrDuplicateKey = errors.New("duplicate key")

ErrDuplicateKey error when a record is inserted and conflicts with an existing record

View Source 
var ErrNoResults = errors.New("no results found")

ErrNoResults error when no results are found

View Source 
var ErrNoSourceFound = errors.New("no source database found in all given configurations")

ErrNoSourceFound is when no source database is found in all given configurations

View Source 
var ErrNotImplemented = errors.New("not implemented")

ErrNotImplemented is an error when a method is not implemented

View Source 
var ErrUnknownCollection = errors.New("could not determine collection name from model")

ErrUnknownCollection is thrown when the collection cannot be found using the model/name

View Source 
var ErrUnknownSQL = errors.New("unknown sql implementation")

ErrUnknownSQL is an error when using a SQL engine that is not known for indexes and migrations

View Source 
var ErrUnsupportedDriver = errors.New("sql driver unsupported")

ErrUnsupportedDriver is when the given SQL driver is not determined to be known or supported

View Source 
var ErrUnsupportedEngine = errors.New("unsupported datastore engine")

ErrUnsupportedEngine is used when the engine given is not a known datastore engine

Functions

func GetModelBoolAttribute 

func GetModelBoolAttribute(model any, attribute string) *bool

GetModelBoolAttribute the attribute from the model as a bool

func GetModelName 

func GetModelName(model any) *string

GetModelName get the name of the model via reflection

func GetModelStringAttribute 

func GetModelStringAttribute(model any, attribute string) *string

GetModelStringAttribute the attribute from the model as a string

func GetModelTableName 

func GetModelTableName(model any) *string

GetModelTableName get the db table name of the model via reflection

func GetModelType 

func GetModelType(model any) reflect.Type

GetModelType get the model type of the model interface via reflection

func GetModelUnset 

func GetModelUnset(model any) map[string]bool

GetModelUnset gets any empty values on the model and makes sure the update actually unsets those values in the database, otherwise this never happens, and we cannot unset

func IsModelSlice 

func IsModelSlice(model any) bool

IsModelSlice returns true if the given interface is a slice of models

func IsSQLEngine 

func IsSQLEngine(e Engine) bool

IsSQLEngine check whether the string already is in the slice

func MarshalQueryParams 

func MarshalQueryParams(m QueryParams) graphql.Marshaler

MarshalQueryParams will marshal the QueryParams struct into a GraphQL marshaler. If all fields of the QueryParams struct are empty or zero, it returns a GraphQL null value.

Parameters: - m: The QueryParams struct to be marshaled.

Returns: - A graphql.Marshaler that represents the marshaled QueryParams struct.

The function performs the following steps: 1. Checks if all fields of the QueryParams struct are empty or zero. 2. If all fields are empty or zero, returns graphql.Null. 3. Otherwise, marshals the QueryParams struct into a generic GraphQL marshaler using graphql.MarshalAny.

func StringInSlice 

func StringInSlice(a string, list []string) bool

StringInSlice check whether the string already is in the slice

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is the datastore client (configuration)

func (*Client) AutoMigrateDatabase 

func (c *Client) AutoMigrateDatabase(ctx context.Context, models ...any) error

AutoMigrateDatabase will detect the database engine and perform the necessary migrations for the provided models. It supports MySQL, PostgreSQL, SQLite, and MongoDB. The function first checks if auto-migration is enabled in the client options. If not enabled, it logs a debug message and returns without performing any migration.

For each model provided, it checks if the model has already been migrated to avoid duplicate migrations. If a model has already been migrated, it returns an error. Otherwise, it adds the model to the list of migrated models.

Depending on the database engine, it performs the migration using the appropriate method:

  • For MongoDB, it calls `autoMigrateMongoDatabase` to create indexes as needed.
  • For SQL databases (MySQL, PostgreSQL, SQLite), it calls `autoMigrateSQLDatabase` using GORM to create or update the table schema.

The function logs debug information about the migration process, including the database engine and the models being migrated.

Parameters: - ctx: The context for the migration process, used for logging and tracing. - models: A variadic list of models to be migrated.

Returns: - An error if the migration fails or if a model has already been migrated.

func (*Client) Close 

func (c *Client) Close(ctx context.Context) error

Close will terminate (close) the datastore and any open connections

func (*Client) CreateInBatches 

func (c *Client) CreateInBatches(
	ctx context.Context,
	models any,
	batchSize int,
) error

CreateInBatches creates all the models given in batches, supporting both SQL and MongoDB engines. For MongoDB, it uses a session context for transaction support if available. For SQL databases, it uses GORM to perform the batch creation.

Parameters: - ctx: The context for the batch creation operation, used for logging and tracing. - models: A slice of models to be created in batches. - batchSize: The number of models to include in each batch.

Returns: - An error if the batch creation operation fails.

The function performs the following steps: 1. Checks the database engine and handles MongoDB separately as it does not support transactions. 2. For SQL databases, it uses GORM's CreateInBatches method to insert the models in batches. 3. Returns any errors encountered during the batch creation operation.

func (*Client) CreateInBatchesMongo 

func (c *Client) CreateInBatchesMongo(
	ctx context.Context,
	models any,
	batchSize int,
) error

CreateInBatchesMongo insert multiple models vai bulk.Write

func (*Client) CustomWhere 

func (c *Client) CustomWhere(tx CustomWhereInterface, conditions map[string]any, engine Engine) any

CustomWhere add conditions

func (*Client) Debug 

func (c *Client) Debug(on bool)

Debug will set the debug flag

func (*Client) DebugLog 

func (c *Client) DebugLog(ctx context.Context, text string)

DebugLog will display verbose logs

func (*Client) Engine 

func (c *Client) Engine() Engine

Engine will return the client's engine

func (*Client) Execute 

func (c *Client) Execute(query string) *gorm.DB

Execute a SQL query (always uses the write database when using dbresolver)

func (*Client) GetArrayFields 

func (c *Client) GetArrayFields() []string

GetArrayFields will return the array fields

func (*Client) GetDatabaseName 

func (c *Client) GetDatabaseName() string

GetDatabaseName will return the full database name for the given model name

func (*Client) GetModel 

func (c *Client) GetModel(
	ctx context.Context,
	model any,
	conditions map[string]any,
	timeout time.Duration,
	forceWriteDB bool,
) error

GetModel will retrieve a single model from the datastore based on the provided conditions. It supports both SQL and MongoDB engines. For MongoDB, it uses a session context for transaction support if available. For SQL databases, it uses GORM to perform the query.

Parameters: - ctx: The context for the retrieval operation, used for logging and tracing. - model: A pointer to the model to be retrieved. - conditions: A map of conditions to filter the query. - timeout: The duration to wait before timing out the query. - forceWriteDB: A boolean indicating whether to force the query to use the "write database" (only applicable for MySQL and PostgreSQL).

Returns: - An error if the retrieval operation fails or if no results are found.

The function performs the following steps: 1. Checks the database engine and handles MongoDB separately as it does not support transactions. 2. Sets the NewRelic transaction to the GORM database if using SQL. 3. Creates a new context and database transaction with the specified timeout. 4. Constructs the query based on the provided conditions and executes it. 5. If forceWriteDB is true, it uses the "write database" for the query (only for MySQL and PostgreSQL). 6. Returns any errors encountered during the retrieval operation or if no results are found.

func (*Client) GetModelCount 

func (c *Client) GetModelCount(
	ctx context.Context,
	model any,
	conditions map[string]any,
	timeout time.Duration,
) (int64, error)

GetModelCount will return a count of the models matching the provided conditions. It supports both SQL and MongoDB engines. For MongoDB, it uses a session context for transaction support if available. For SQL databases, it uses GORM to perform the count operation.

Parameters: - ctx: The context for the count operation, used for logging and tracing. - model: A pointer to the model type for which the count is to be retrieved. - conditions: A map of conditions to filter the count query. - timeout: The duration to wait before timing out the query.

Returns: - count: The number of models matching the provided conditions. - err: An error if the count operation fails.

The function performs the following steps: 1. Checks the database engine and handles MongoDB separately as it does not support transactions. 2. Sets the NewRelic transaction to the GORM database if using SQL. 3. Creates a new context and database transaction with the specified timeout. 4. Constructs the count query based on the provided conditions and executes it. 5. Returns the count of models and any errors encountered during the count operation.

func (*Client) GetModelPartial added in v1.2.0 

func (c *Client) GetModelPartial(
	ctx context.Context,
	model any,
	fieldResult any,
	conditions map[string]any,
	timeout time.Duration,
	forceWriteDB bool,
) error

GetModelPartial will retrieve a single model from the datastore based on the provided conditions and selects specific fields. It supports both SQL and MongoDB engines. For MongoDB, it uses a session context for transaction support if available. For SQL databases, it uses GORM to perform the query.

Parameters: - ctx: The context for the retrieval operation, used for logging and tracing. - model: A pointer to the model to be retrieved (used for table/collection name). - fieldResult: A pointer to the struct where the results will be stored. - conditions: A map of conditions to filter the query. - timeout: The duration to wait before timing out the query. - forceWriteDB: A boolean indicating whether to force the query to use the "write database" (only applicable for MySQL and PostgreSQL).

Returns: - An error if the retrieval operation fails or if no results are found.

func (*Client) GetModels 

func (c *Client) GetModels(
	ctx context.Context,
	models any,
	conditions map[string]any,
	queryParams *QueryParams,
	fieldResults any,
	timeout time.Duration,
) error

GetModels will return a slice of models based on the given conditions and query parameters. It supports both SQL and MongoDB engines. For MongoDB, it uses a session context for transaction support if available. For SQL databases, it uses GORM to perform the query.

Parameters: - ctx: The context for the retrieval operation, used for logging and tracing. - models: A pointer to a slice of models to be retrieved. - conditions: A map of conditions to filter the query. - queryParams: A pointer to QueryParams struct containing pagination and sorting information. - fieldResults: A pointer to a slice where the results will be stored if not nil. - timeout: The duration to wait before timing out the query.

Returns: - An error if the retrieval operation fails or if no results are found.

The function performs the following steps: 1. Initializes default values for queryParams if not provided. 2. Checks the database engine and handles MongoDB separately as it does not support transactions. 3. Sets the NewRelic transaction to the GORM database if using SQL. 4. Creates a new context and database transaction with the specified timeout. 5. Constructs the query based on the provided conditions, pagination, and sorting information. 6. Executes the query and stores the results in the provided models or fieldResults slice. 7. Returns any errors encountered during the retrieval operation or if no results are found.

func (*Client) GetModelsAggregate 

func (c *Client) GetModelsAggregate(ctx context.Context, models any,
	conditions map[string]any, aggregateColumn string, timeout time.Duration,
) (map[string]any, error)

GetModelsAggregate will return an aggregate count of the model matching conditions. It supports both SQL and MongoDB engines. For MongoDB, it uses a session context for transaction support if available. For SQL databases, it uses GORM to perform the aggregate operation.

Parameters: - ctx: The context for the aggregate operation, used for logging and tracing. - models: A pointer to a slice of models to be aggregated. - conditions: A map of conditions to filter the aggregate query. - aggregateColumn: The name of the column to aggregate. - timeout: The duration to wait before timing out the query.

Returns: - result: A map where the keys are the aggregated column values and the values are the counts of models matching the conditions. - err: An error if the aggregate operation fails.

The function performs the following steps: 1. Checks the database engine and handles MongoDB separately as it does not support transactions. 2. Sets the NewRelic transaction to the GORM database if using SQL. 3. Creates a new context and database transaction with the specified timeout. 4. Constructs the aggregate query based on the provided conditions and executes it. 5. For date fields, formats the date according to the database engine. 6. Returns the aggregate result and any errors encountered during the aggregate operation.

func (*Client) GetModelsPartial added in v1.2.0 

func (c *Client) GetModelsPartial(
	ctx context.Context,
	models any,
	fieldResults any,
	conditions map[string]any,
	timeout time.Duration,
) error

GetModelsPartial will retrieve multiple models from the datastore based on the provided conditions and selects specific fields. It supports both SQL and MongoDB engines. For MongoDB, it uses a session context for transaction support if available. For SQL databases, it uses GORM to perform the query.

Parameters: - ctx: The context for the retrieval operation, used for logging and tracing. - models: A pointer to a slice of models to be retrieved (used for table/collection name). - fieldResults: A pointer to a slice where the results will be stored, or []string of field names to select. - conditions: A map of conditions to filter the query. - timeout: The duration to wait before timing out the query.

Returns: - An error if the retrieval operation fails or if no results are found.

The fieldResults parameter supports three modes: 1. []string: list of field names to select (destination = models) 2. struct slice pointer: destination struct slice (GORM auto-selects based on struct fields) 3. nil: select all fields (destination = models)

func (*Client) GetMongoCollection 

func (c *Client) GetMongoCollection(
	collectionName string,
) *mongo.Collection

GetMongoCollection will get the mongo collection for the given tableName

func (*Client) GetMongoCollectionByTableName 

func (c *Client) GetMongoCollectionByTableName(
	tableName string,
) *mongo.Collection

GetMongoCollectionByTableName will get the mongo collection for the given tableName

func (*Client) GetMongoConditionProcessor 

func (c *Client) GetMongoConditionProcessor() func(conditions *map[string]any)

GetMongoConditionProcessor will return a custom mongo condition processor if set

func (*Client) GetMongoIndexer 

func (c *Client) GetMongoIndexer() func() map[string][]mongo.IndexModel

GetMongoIndexer will return a custom mongo condition indexer

func (*Client) GetObjectFields 

func (c *Client) GetObjectFields() []string

GetObjectFields will return the object fields

func (*Client) GetTableName 

func (c *Client) GetTableName(modelName string) string

GetTableName will return the full table name for the given model name

func (*Client) HasMigratedModel 

func (c *Client) HasMigratedModel(modelType string) bool

HasMigratedModel will return if the model type has been migrated

func (*Client) IncrementModel 

func (c *Client) IncrementModel(
	ctx context.Context,
	model any,
	fieldName string,
	increment int64,
) (newValue int64, err error)

IncrementModel will increment the given field atomically in the database and return the new value. It supports both SQL and MongoDB engines. For MongoDB, it uses a session context for transaction support if available. For SQL databases, it uses GORM to perform the increment operation within a transaction.

Parameters: - ctx: The context for the increment operation, used for logging and tracing. - model: A pointer to the model to be incremented. - fieldName: The name of the field to be incremented. - increment: The value by which to increment the field.

Returns: - newValue: The new value of the incremented field. - err: An error if the increment operation fails.

The function performs the following steps: 1. Checks the database engine and handles MongoDB separately as it does not support transactions. 2. Sets the NewRelic transaction to the GORM database if using SQL. 3. Creates a new transaction and locks the row for update to ensure atomicity. 4. Retrieves the current value of the field and increments it by the specified amount. 5. Updates the field with the new value in the database. 6. Returns the new value and any errors encountered during the increment operation.

func (*Client) IndexExists 

func (c *Client) IndexExists(tableName, indexName string) (bool, error)

IndexExists check whether the given index exists in the datastore

func (*Client) IndexMetadata 

func (c *Client) IndexMetadata(tableName, field string) error

IndexMetadata check and create the metadata JSON index

func (*Client) IsAutoMigrate 

func (c *Client) IsAutoMigrate() bool

IsAutoMigrate returns whether auto migration is on

func (*Client) IsDebug 

func (c *Client) IsDebug() bool

IsDebug will return the debug flag (bool)

func (*Client) IsNewRelicEnabled 

func (c *Client) IsNewRelicEnabled() bool

IsNewRelicEnabled will return if new relic is enabled

func (*Client) NewRawTx added in v0.4.0 

func (c *Client) NewRawTx() (*Transaction, error)

NewRawTx will start a new datastore transaction based on the configured database options. It supports both GORM-based SQL databases and MongoDB, handling the transaction lifecycle accordingly.

Returns: - Transaction: A pointer to the started Transaction struct. - error: An error if the transaction initialization fails.

The function performs the following steps: 1. Checks if a GORM database is configured. If so, it starts a new GORM session and begins a transaction. 2. If MongoDB transactions are enabled, it returns an error as MongoDB transactions require a callback function. 3. If no database is configured, it returns an empty Transaction struct.

func (*Client) NewTx 

func (c *Client) NewTx(ctx context.Context, fn func(*Transaction) error) error

NewTx will start a new datastore transaction based on the configured database options. It supports both GORM-based SQL databases and MongoDB, handling the transaction lifecycle accordingly.

Parameters: - ctx: The context for the transaction, used for managing request-scoped values, cancellation signals, and deadlines. - fn: A function that takes a pointer to a Transaction and returns an error. This function contains the operations to be performed within the transaction.

Returns: - error: An error if the transaction initialization or the provided function fails.

The function performs the following steps: 1. Checks if a GORM database is configured. If so, it starts a new GORM session and begins a transaction. 2. If MongoDB transactions are enabled, it starts a new MongoDB session and transaction. 3. If no database is configured, it executes the provided function with an empty transaction. 4. The provided function is executed within the context of the started transaction.

func (*Client) Raw 

func (c *Client) Raw(query string) *gorm.DB

Raw a raw SQL query

func (*Client) SaveModel 

func (c *Client) SaveModel(
	ctx context.Context,
	model any,
	tx *Transaction,
	newRecord, commitTx bool,
) error

SaveModel will handle creating or updating a model based on its primary key, abstracting the database operations. It supports both SQL and MongoDB engines. For MongoDB, it uses a session context for transaction support if available. For SQL databases, it uses GORM to create or update the table schema.

Parameters: - ctx: The context for the save operation, used for logging and tracing. - model: A pointer to the model to be saved. - tx: The transaction object to be used for the save operation. - newRecord: A boolean indicating whether the model is a new record (true) or an existing record (false). - commitTx: A boolean indicating whether to commit the transaction after saving the model.

Returns: - An error if the save operation fails.

The function performs the following steps: 1. Checks the database engine and handles MongoDB separately as it does not support transactions. 2. Sets the NewRelic transaction to the GORM database if using SQL. 3. Captures any panics during the save operation and rolls back the transaction if a panic occurs. 4. For new records, it creates the model in the database, omitting associations. 5. For existing records, it updates the model in the database, omitting associations. 6. Commits the transaction if commitTx is true. 7. Returns any errors encountered during the save operation.

type ClientInterface 

type ClientInterface interface {
	GetterInterface
	StorageService
	Close(ctx context.Context) error
	Debug(on bool)
	DebugLog(ctx context.Context, text string)
	Engine() Engine
	IsAutoMigrate() bool
	IsDebug() bool
	IsNewRelicEnabled() bool
}

ClientInterface is the Datastore client interface

func NewClient 

func NewClient(ctx context.Context, opts ...ClientOps) (ClientInterface, error)

NewClient creates a new client for all Datastore functionality

If no options are given, it will use the defaultClientOptions() ctx may contain a NewRelic txn (or one will be created)

type ClientOps 

type ClientOps func(c *clientOptions)

ClientOps allow functional options to be supplied that overwrite default client options.

func WithAutoMigrate 

func WithAutoMigrate(migrateModels ...any) ClientOps

WithAutoMigrate will enable auto migrate database mode (given models)

Pointers of structs (IE: &models.Xpub{})

func WithCustomFields 

func WithCustomFields(arrayFields, objectFields []string) ClientOps

WithCustomFields will add custom fields to the datastore

func WithCustomMongoConditionProcessor 

func WithCustomMongoConditionProcessor(f func(conditions *map[string]any)) ClientOps

WithCustomMongoConditionProcessor will add a custom mongo condition processor function

func WithCustomMongoIndexer 

func WithCustomMongoIndexer(f func() map[string][]mongo.IndexModel) ClientOps

WithCustomMongoIndexer will add a custom mongo index function (returns custom mongo indexes)

func WithDebugging 

func WithDebugging() ClientOps

WithDebugging will enable debugging mode

func WithLogger 

func WithLogger(customLogger zLogger.GormLoggerInterface) ClientOps

WithLogger will set the custom logger interface

func WithMongo 

func WithMongo(config *MongoDBConfig) ClientOps

WithMongo will set the datastore to use MongoDB

func WithMongoConnection 

func WithMongoConnection(database *mongo.Database, tablePrefix string) ClientOps

WithMongoConnection will set the datastore to use an existing Mongo database connection

func WithNewRelic 

func WithNewRelic() ClientOps

WithNewRelic will enable the NewRelic wrapper

func WithSQL 

func WithSQL(engine Engine, configs []*SQLConfig) ClientOps

WithSQL will load a datastore using either an SQL database config or existing connection

func WithSQLConnection 

func WithSQLConnection(engine Engine, sqlDB *sql.DB, tablePrefix string) ClientOps

WithSQLConnection will set the datastore to an existing connection for MySQL or PostgreSQL

func WithSQLite 

func WithSQLite(config *SQLiteConfig) ClientOps

WithSQLite will set the datastore to use SQLite

type CommonConfig 

type CommonConfig struct {
	Debug                 bool          `json:"debug" mapstructure:"debug"`                                       // flag for debugging sql queries in logs
	MaxConnectionIdleTime time.Duration `json:"max_connection_idle_time" mapstructure:"max_connection_idle_time"` // 360
	MaxConnectionTime     time.Duration `json:"max_connection_time" mapstructure:"max_connection_time"`           // 60
	MaxIdleConnections    int           `json:"max_idle_connections" mapstructure:"max_idle_connections"`         // 5
	MaxOpenConnections    int           `json:"max_open_connections" mapstructure:"max_open_connections"`         // 5
	TablePrefix           string        `json:"table_prefix" mapstructure:"table_prefix"`                         // pre_users (pre)
}

CommonConfig is the common configuration fields between engines

type CustomWhereInterface 

type CustomWhereInterface interface {
	Where(query any, args ...any)
	// contains filtered or unexported methods
}

CustomWhereInterface is an interface for the CustomWhere clauses

type DatabaseLogWrapper 

type DatabaseLogWrapper struct {
	zLogger.GormLoggerInterface
}

DatabaseLogWrapper is a special wrapper for the GORM logger

func (*DatabaseLogWrapper) LogMode 

func (d *DatabaseLogWrapper) LogMode(level gLogger.LogLevel) gLogger.Interface

LogMode will set the log level/mode

type Engine 

type Engine string

Engine is the different engines that are supported (database) 

const (
	Empty      Engine = "empty"
	MongoDB    Engine = "mongodb"
	MySQL      Engine = "mysql"
	PostgreSQL Engine = "postgresql"
	SQLite     Engine = "sqlite"
)

Supported engines (databases)

func (Engine) IsEmpty 

func (e Engine) IsEmpty() bool

IsEmpty will return true if the datastore is not set

func (Engine) String 

func (e Engine) String() string

String is the string version of engine

type GetterInterface 

type GetterInterface interface {
	GetArrayFields() []string
	GetDatabaseName() string
	GetMongoCollection(collectionName string) *mongo.Collection
	GetMongoCollectionByTableName(tableName string) *mongo.Collection
	GetMongoConditionProcessor() func(conditions *map[string]any)
	GetMongoIndexer() func() map[string][]mongo.IndexModel
	GetObjectFields() []string
	GetTableName(modelName string) string
}

GetterInterface is the getter methods

type MongoDBConfig 

type MongoDBConfig struct {
	CommonConfig `json:",inline" mapstructure:",squash"` // Common configuration

	DatabaseName       string          `json:"database_name" mapstructure:"database_name"` // The database name
	ExistingConnection *mongo.Database `json:"-" mapstructure:"-"`                         // Used for existing database connection
	Transactions       bool            `json:"transactions" mapstructure:"transactions"`   // If it has transactions
	URI                string          `json:"uri" mapstructure:"uri"`                     // The connection string URI
}

MongoDBConfig is the configuration for each MongoDB connection

type QueryParams 

type QueryParams struct {
	Page          int    `json:"page,omitempty"`
	PageSize      int    `json:"page_size,omitempty"`
	OrderByField  string `json:"order_by_field,omitempty"`
	SortDirection string `json:"sort_direction,omitempty"`
}

QueryParams object to use when limiting and sorting database query results

func UnmarshalQueryParams 

func UnmarshalQueryParams(v any) (QueryParams, error)

UnmarshalQueryParams will unmarshal the provided interface into a QueryParams struct. It handles the conversion from a generic interface to the specific QueryParams type, ensuring that the data is correctly parsed and assigned to the struct fields.

Parameters: - v: The any to be unmarshalled. It is expected to be a map or a JSON-like structure.

Returns: - QueryParams: The unmarshalled QueryParams struct with the parsed data. - error: An error if the unmarshalling process fails.

The function performs the following steps: 1. Checks if the provided interface is nil, returning an empty QueryParams struct if true. 2. Marshals the interface into a JSON byte slice. 3. Unmarshal the JSON byte slice into a QueryParams struct. 4. Returns the populated QueryParams struct and any error encountered during the process.

type SQLConfig 

type SQLConfig struct {
	CommonConfig `json:",inline" mapstructure:",squash"` // Common configuration

	Driver                    string        `json:"driver" mapstructure:"driver"`                                             // mysql or postgresql
	ExistingConnection        *sql.DB       `json:"-" mapstructure:"-"`                                                       // Used for existing database connection
	Host                      string        `json:"host" mapstructure:"host"`                                                 // database host IE: localhost
	Name                      string        `json:"name" mapstructure:"name"`                                                 // database-name
	Password                  string        `json:"password" mapstructure:"password" encrypted:"true"`                        // user-password (field is encrypted at rest)
	Port                      string        `json:"port" mapstructure:"port"`                                                 // 3306
	Replica                   bool          `json:"replica" mapstructure:"replica"`                                           // True if it's a replica (Read-Only)
	SkipInitializeWithVersion bool          `json:"skip_initialize_with_version" mapstructure:"skip_initialize_with_version"` // Skip using MySQL in test mode
	TimeZone                  string        `json:"time_zone" mapstructure:"time_zone"`                                       // timezone (IE: Asia/Shanghai)
	TxTimeout                 time.Duration `json:"tx_timeout" mapstructure:"tx_timeout"`                                     // 5*time.Second
	User                      string        `json:"user" mapstructure:"user"`                                                 // database username
	SslMode                   string        `json:"ssl_mode" mapstructure:"ssl_mode"`                                         // ssl mode (for PostgreSQL) [disable|allow|prefer|require|verify-ca|verify-full]
}

SQLConfig is the configuration for each SQL connection (mysql or postgresql)

type SQLiteConfig 

type SQLiteConfig struct {
	CommonConfig `json:",inline" mapstructure:",squash"` // Common configuration

	DatabasePath       string        `json:"database_path" mapstructure:"database_path"` // Location of a permanent database file (if NOT set, uses temporary memory)
	ExistingConnection gorm.ConnPool `json:"-" mapstructure:"-"`                         // Used for existing database connection
	Shared             bool          `json:"shared" mapstructure:"shared"`               // Adds a shared param to the connection string
}

SQLiteConfig is the configuration for each SQLite connection

type StorageService 

type StorageService interface {
	AutoMigrateDatabase(ctx context.Context, models ...any) error
	CreateInBatches(ctx context.Context, models any, batchSize int) error
	CustomWhere(tx CustomWhereInterface, conditions map[string]any, engine Engine) any
	Execute(query string) *gorm.DB
	GetModel(ctx context.Context, model any, conditions map[string]any,
		timeout time.Duration, forceWriteDB bool) error
	GetModelPartial(ctx context.Context, model, fieldResult any, conditions map[string]any,
		timeout time.Duration, forceWriteDB bool) error
	GetModels(ctx context.Context, models any, conditions map[string]any, queryParams *QueryParams,
		fieldResults any, timeout time.Duration) error
	GetModelsPartial(ctx context.Context, models, fieldResults any, conditions map[string]any,
		timeout time.Duration) error
	GetModelCount(ctx context.Context, model any, conditions map[string]any,
		timeout time.Duration) (int64, error)
	GetModelsAggregate(ctx context.Context, models any, conditions map[string]any,
		aggregateColumn string, timeout time.Duration) (map[string]any, error)
	HasMigratedModel(modelType string) bool
	IncrementModel(ctx context.Context, model any,
		fieldName string, increment int64) (newValue int64, err error)
	IndexExists(tableName, indexName string) (bool, error)
	IndexMetadata(tableName, field string) error
	NewTx(ctx context.Context, fn func(*Transaction) error) error
	NewRawTx() (*Transaction, error)
	Raw(query string) *gorm.DB
	SaveModel(ctx context.Context, model any, tx *Transaction, newRecord, commitTx bool) error
}

StorageService is the storage-related methods

type Transaction 

type Transaction struct {
	// contains filtered or unexported fields
}

Transaction is the internal datastore transaction

func (*Transaction) CanCommit 

func (tx *Transaction) CanCommit() bool

CanCommit will return true if it can commit

func (*Transaction) Commit 

func (tx *Transaction) Commit() error

Commit will commit the transaction

func (*Transaction) Rollback 

func (tx *Transaction) Rollback() error

Rollback the transaction

Source Files

View all Source files

Directories

Path Synopsis
Package customtypes encapsulates our custom database field types
 Package customtypes encapsulates our custom database field types
Package nrgorm integrates New Relic into the GORM database layer
 Package nrgorm integrates New Relic into the GORM database layer

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.