mdbstore

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 28, 2026 License: MPL-2.0, MIT Imports: 22 Imported by: 0

README

mdbstore

mdbstore is a MariaDB-backed session store for Hypermeow. It implements Hypermeow's device container and all session-specific storage interfaces for applications that need durable WhatsApp session state in MariaDB.

The module path is github.com/ReindeerGames/mdbstore.

Hypermeow is an unofficial WhatsApp client. This project is not affiliated with, authorized by, or endorsed by WhatsApp or Meta.

Compatibility

  • Go 1.26.10 or newer. Earlier Go 1.26 patch releases contain standard-library vulnerabilities reachable through TLS certificate verification.
  • MariaDB 11.4 LTS is continuously tested. MySQL is intentionally rejected because its behavior is not part of this adapter's compatibility contract.
  • Hypermeow commit 7852ff1c171870a1fe43ce366799a00385572fd9 is pinned and compile-time interface assertions protect the complete store surface.

The schema covers device credentials, Signal identities, sessions, prekeys, sender keys, app state, contacts, chat settings, message secrets, privacy tokens, NCT salt, LID mappings, decryption buffering, and retry buffering.

Repository layout

The module root is the public mdbstore package. Its files are grouped by the API or Hypermeow store area they implement. Private implementation details are kept below internal/:

  • internal/database translates Hypermeow's SQL parameters, manages transactions, and records redacted storage telemetry.
  • internal/schema owns schema validation and embedded, versioned MariaDB migrations.

This keeps the public import path stable and prevents applications from depending on migration or SQL-execution internals.

Installation

go get github.com/ReindeerGames/mdbstore

Usage

package main

import (
	"context"
	"log"

	"github.com/ReindeerGames/mdbstore"
	whatsmeow "github.com/polymorfa/hypermeow"
)

func main() {
	ctx := context.Background()
	container, err := mdbstore.New(
		ctx,
		"whatsapp:secret@tcp(db.example:3306)/whatsapp?tls=true",
		nil,
	)
	if err != nil {
		log.Fatal(err)
	}
	defer container.Close()

	device, err := container.GetFirstDevice(ctx)
	if err != nil {
		log.Fatal(err)
	}
	client := whatsmeow.NewClient(device, nil)
	_ = client
}

New parses the standard go-sql-driver/mysql DSN, adds bounded network timeouts when they are omitted, configures the connection pool, verifies that the server is MariaDB, and applies schema migrations under a database-qualified advisory lock.

Use NewWithConfig to change pool and migration settings. Start with NewDefaultConfig; a zero Config also selects all defaults. In a non-zero configuration, zero idle connections and zero connection lifetimes are valid and disable those features; use negative values to restore their defaults.

NewWithDB borrows an existing *sql.DB without changing its pool settings. NewWithDBConfig borrows it and applies the supplied pool settings. Neither constructor takes ownership: call Upgrade before use and close the pool in the application. Container.Close only closes pools created by New or NewWithConfig.

Database setup

Use a dedicated database and account. The account needs DDL permission while migrations run and DML permission while the service runs. Deployments that separate migration and runtime identities can call Upgrade during a privileged release step, then construct a borrowed container with a DML-only account and call Validate. Validate performs the complete server, version, and structural checks without executing DDL.

Production connections should use TLS with certificate verification. Keep DSNs in a secret manager and never log them. MariaDB should use InnoDB and a durability policy appropriate to the application's recovery requirements.

Migrations are embedded in the module. Every startup validates required tables, columns, primary keys, indexes, cascading foreign keys, key-material types, and exact-length constraints. A partial or incompatible schema fails closed rather than being accepted because a version row exists.

Concurrency and consistency

The connection pool and store implementations are safe for concurrent use. Deadlocks and lock-wait timeouts are retried up to three times with bounded backoff.

Run only one active Hypermeow client for a given device. This is a WhatsApp session ownership requirement, not a MariaDB limitation. Multiple processes may host different devices in the same database. Global LID mappings are read authoritatively from MariaDB so updates made by another process are visible.

Large reads, writes, and deletes are chunked to remain below MariaDB parameter and packet limits while transactionally preserving operations that must be atomic.

Health and observability

  • Ping measures database reachability.
  • SchemaVersion reports the installed adapter schema version.
  • DBStats exposes the standard connection-pool statistics.
  • Metrics returns cumulative operation, error, latency, transaction, retry, rollback, commit, and observer-panic counters.
  • Config.Observer receives redacted operation events suitable for adapting to Prometheus or OpenTelemetry. Events contain safe logical operation names, duration, failure class, and MariaDB error number; SQL, arguments, session data, and raw database errors are excluded.

Observers run synchronously and must return promptly. They must not call back into the same container. Panics are recovered and exposed in Metrics.

Recommended alerts include database errors, pool waits, failed or retried transactions, connection saturation, schema validation failures, InnoDB deadlocks, lock waits, storage growth, backup failure, and repeated client disconnect or logout events.

Testing

Fast checks:

go test -race ./...
go vet ./...
go run golang.org/x/vuln/cmd/govulncheck@latest ./...

The MariaDB integration test is enabled by setting MDBSTORE_TEST_DSN:

MDBSTORE_TEST_DSN='user:password@tcp(127.0.0.1:3306)/mdbstore_test' \
  go test -race ./...

Use an isolated disposable database. The integration test exercises migrations and every Hypermeow storage family and removes its device data on success.

Operational guidance

  • Back up MariaDB and regularly prove restoration into a clean server.
  • Monitor binary-log, disk, table, and index growth.
  • Pin explicit mdbstore and Hypermeow versions; review protocol and schema changes before upgrading.
  • Use graceful shutdown so Hypermeow disconnects before the database pool is closed.
  • Never log QR data, pairing codes, session keys, message plaintext, uploaded media, SQL arguments, credentials, or full DSNs.

License

Adapter files derived from Hypermeow's SQL store retain their Mozilla Public License 2.0 notices. Upstream-derived tests covered by MIT terms identify that license explicitly. See LICENSE, LICENSE-MIT, and NOTICE.

Documentation

Overview

Package mdbstore provides a MariaDB-backed implementation of Hypermeow's complete session storage interfaces.

New is the usual entry point. It owns the returned database connection pool, validates the server as MariaDB, and applies schema migrations before use.

Index

Constants

View Source
const LatestSchemaVersion = internalSchema.LatestVersion

LatestSchemaVersion is the schema version required by this adapter release.

Variables

View Source
var ErrDeviceIDMustBeSet = errors.New("device JID must be known before accessing database")

ErrDeviceIDMustBeSet is the error returned by PutDevice if you try to save a device before knowing its JID.

View Source
var ErrInvalidLength = errors.New("database returned byte array with illegal length")

ErrInvalidLength is returned by some database getters if the database returned a byte array with an unexpected length. This should be impossible, as the database schema contains CHECK()s for all the relevant columns.

Functions

This section is empty.

Types

type CachedLIDMap

type CachedLIDMap struct {
	// contains filtered or unexported fields
}

func (*CachedLIDMap) FillCache

func (s *CachedLIDMap) FillCache(ctx context.Context) error

func (*CachedLIDMap) GetLIDForPN

func (s *CachedLIDMap) GetLIDForPN(ctx context.Context, pn types.JID) (types.JID, error)

func (*CachedLIDMap) GetManyLIDsForPNs

func (s *CachedLIDMap) GetManyLIDsForPNs(ctx context.Context, pns []types.JID) (map[types.JID]types.JID, error)

func (*CachedLIDMap) GetManyPNsForLIDs

func (s *CachedLIDMap) GetManyPNsForLIDs(ctx context.Context, lids []types.JID) (map[types.JID]types.JID, error)

func (*CachedLIDMap) GetPNForLID

func (s *CachedLIDMap) GetPNForLID(ctx context.Context, lid types.JID) (types.JID, error)

func (*CachedLIDMap) PutLIDMapping

func (s *CachedLIDMap) PutLIDMapping(ctx context.Context, lid, pn types.JID) error

func (*CachedLIDMap) PutManyLIDMappings

func (s *CachedLIDMap) PutManyLIDMappings(ctx context.Context, mappings []store.LIDMapping) error

type Config

type Config struct {
	MaxOpenConns      int
	MaxIdleConns      int
	ConnMaxLifetime   time.Duration
	ConnMaxIdleTime   time.Duration
	MigrationLockWait time.Duration
	Observer          Observer
}

Config controls connection pooling and schema migration behavior.

func NewDefaultConfig

func NewDefaultConfig() Config

NewDefaultConfig returns an independent configuration suitable for a service hosting multiple active sessions. Returning a copy prevents global configuration mutations and data races between callers.

type Container

type Container struct {
	LIDMap *CachedLIDMap
	// contains filtered or unexported fields
}

Container is a wrapper for a SQL database that can contain multiple whatsmeow sessions.

func New

func New(ctx context.Context, dsn string, log waLog.Logger) (*Container, error)

New opens a MariaDB-backed Hypermeow session container and applies pending schema migrations. The DSN uses github.com/go-sql-driver/mysql syntax.

func NewWithConfig

func NewWithConfig(ctx context.Context, dsn string, log waLog.Logger, config Config) (*Container, error)

NewWithConfig is New with explicit connection-pool and migration settings.

func NewWithDB

func NewWithDB(db *sql.DB, log waLog.Logger) *Container

NewWithDB borrows an existing MariaDB connection pool without changing its settings. It does not run migrations; call Upgrade before serving clients. Close does not close a borrowed pool; the caller retains ownership.

func NewWithDBConfig

func NewWithDBConfig(db *sql.DB, log waLog.Logger, config Config) *Container

NewWithDBConfig borrows an existing pool and applies the explicit pool settings. Close does not close the borrowed pool.

func (*Container) Close

func (c *Container) Close() error

Close closes pools opened by New. Borrowed pools passed to NewWithDB or NewWithDBConfig remain owned by their caller.

func (*Container) DBStats

func (c *Container) DBStats() sql.DBStats

DBStats returns a snapshot of the underlying database/sql connection pool.

func (*Container) DeleteDevice

func (c *Container) DeleteDevice(ctx context.Context, store *store.Device) error

DeleteDevice deletes the given device from this database. This should be called through Device.Delete()

func (*Container) GetAllDevices

func (c *Container) GetAllDevices(ctx context.Context) ([]*store.Device, error)

GetAllDevices finds all the devices in the database.

func (*Container) GetDevice

func (c *Container) GetDevice(ctx context.Context, jid types.JID) (*store.Device, error)

GetDevice finds the device with the specified JID in the database.

If the device is not found, nil is returned instead.

Note that the parameter usually must be an AD-JID.

func (*Container) GetFirstDevice

func (c *Container) GetFirstDevice(ctx context.Context) (*store.Device, error)

GetFirstDevice is a convenience method for getting the first device in the store. If there are no devices, then a new device will be created. You should only use this if you don't want to have multiple sessions simultaneously.

func (*Container) Metrics

func (c *Container) Metrics() MetricsSnapshot

Metrics returns cumulative, redacted storage-operation telemetry.

func (*Container) NewDevice

func (c *Container) NewDevice() *store.Device

NewDevice creates a new device in this database.

No data is actually stored before Save is called. However, the pairing process will automatically call Save after a successful pairing, so you most likely don't need to call it yourself.

func (*Container) Ping

func (c *Container) Ping(ctx context.Context) error

Ping verifies that MariaDB is reachable.

func (*Container) PutDevice

func (c *Container) PutDevice(ctx context.Context, device *store.Device) error

PutDevice stores the given device in this database. This should be called through Device.Save() (which usually doesn't need to be called manually, as the library does that automatically when relevant).

func (*Container) SchemaVersion

func (c *Container) SchemaVersion(ctx context.Context) (int, error)

SchemaVersion returns the adapter schema version currently stored in MariaDB.

func (*Container) Upgrade

func (c *Container) Upgrade(ctx context.Context) error

Upgrade upgrades the database from its current schema to the latest version.

func (*Container) Validate

func (c *Container) Validate(ctx context.Context) error

Validate verifies the MariaDB server, installed schema version, and complete schema structure without applying DDL. It is suitable for startup checks performed with a DML-only runtime identity after a privileged migration step.

type DBEvent

type DBEvent struct {
	Operation  string
	Duration   time.Duration
	Failed     bool
	ErrorClass string
	ErrorCode  uint16
}

DBEvent is a redacted storage-operation measurement. SQL text and arguments are intentionally excluded because session data is sensitive.

type MetricsSnapshot

type MetricsSnapshot struct {
	Operations           uint64        `json:"operations"`
	Errors               uint64        `json:"errors"`
	TotalLatency         time.Duration `json:"total_latency"`
	Transactions         uint64        `json:"transactions"`
	TransactionCommits   uint64        `json:"transaction_commits"`
	TransactionRollbacks uint64        `json:"transaction_rollbacks"`
	TransactionErrors    uint64        `json:"transaction_errors"`
	TransactionRetries   uint64        `json:"transaction_retries"`
	ObserverPanics       uint64        `json:"observer_panics"`
}

MetricsSnapshot is a lock-free cumulative storage telemetry snapshot.

type Observer

type Observer interface {
	ObserveDB(context.Context, DBEvent)
}

Observer receives completed storage-operation measurements synchronously. Implementations must return promptly and must not call back into the same Container. Panics are recovered and counted in MetricsSnapshot.

type ObserverFunc

type ObserverFunc func(context.Context, DBEvent)

ObserverFunc adapts a function to Observer.

func (ObserverFunc) ObserveDB

func (fn ObserverFunc) ObserveDB(ctx context.Context, event DBEvent)

type SQLStore

type SQLStore struct {
	*Container
	JID string
	// contains filtered or unexported fields
}

func NewSQLStore

func NewSQLStore(c *Container, jid types.JID) *SQLStore

NewSQLStore creates a new SQLStore with the given database container and user JID. It contains implementations of all the different stores in the store package.

In general, you should use Container.NewDevice or Container.GetDevice instead of this.

func (*SQLStore) AddOutgoingEvent

func (s *SQLStore) AddOutgoingEvent(ctx context.Context, chatJID types.JID, id types.MessageID, format string, plaintext []byte) error

func (*SQLStore) ClearBufferedEventPlaintext

func (s *SQLStore) ClearBufferedEventPlaintext(ctx context.Context, ciphertextHash [32]byte) error

func (*SQLStore) DeleteAllIdentities

func (s *SQLStore) DeleteAllIdentities(ctx context.Context, phone string) error

func (*SQLStore) DeleteAllSessions

func (s *SQLStore) DeleteAllSessions(ctx context.Context, phone string) error

func (*SQLStore) DeleteAppStateMutationMACs

func (s *SQLStore) DeleteAppStateMutationMACs(ctx context.Context, name string, indexMACs [][]byte) (err error)

func (*SQLStore) DeleteAppStateVersion

func (s *SQLStore) DeleteAppStateVersion(ctx context.Context, name string) error

func (*SQLStore) DeleteExpiredPrivacyTokens

func (s *SQLStore) DeleteExpiredPrivacyTokens(ctx context.Context, cutoff time.Time) (int64, error)

func (*SQLStore) DeleteIdentity

func (s *SQLStore) DeleteIdentity(ctx context.Context, address string) error

func (*SQLStore) DeleteNCTSalt

func (s *SQLStore) DeleteNCTSalt(ctx context.Context) error

func (*SQLStore) DeleteOldBufferedHashes

func (s *SQLStore) DeleteOldBufferedHashes(ctx context.Context) error

func (*SQLStore) DeleteOldOutgoingEvents

func (s *SQLStore) DeleteOldOutgoingEvents(ctx context.Context) error

func (*SQLStore) DeleteSession

func (s *SQLStore) DeleteSession(ctx context.Context, address string) error

func (*SQLStore) DoDecryptionTxn

func (s *SQLStore) DoDecryptionTxn(ctx context.Context, fn func(context.Context) error) error

func (*SQLStore) EnsureIdentity

func (s *SQLStore) EnsureIdentity(ctx context.Context, address string, key [32]byte, deleteGeneration uint64) (bool, error)

func (*SQLStore) GenOnePreKey

func (s *SQLStore) GenOnePreKey(ctx context.Context) (*keys.PreKey, error)

func (*SQLStore) GetAllAppStateSyncKeys

func (s *SQLStore) GetAllAppStateSyncKeys(ctx context.Context) ([]*store.AppStateSyncKey, error)

func (*SQLStore) GetAllContacts

func (s *SQLStore) GetAllContacts(ctx context.Context) (map[types.JID]types.ContactInfo, error)

func (*SQLStore) GetAppStateMutationMAC

func (s *SQLStore) GetAppStateMutationMAC(ctx context.Context, name string, indexMAC []byte) (valueMAC []byte, err error)

func (*SQLStore) GetAppStateSyncKey

func (s *SQLStore) GetAppStateSyncKey(ctx context.Context, id []byte) (*store.AppStateSyncKey, error)

func (*SQLStore) GetAppStateVersion

func (s *SQLStore) GetAppStateVersion(ctx context.Context, name string) (version uint64, hash [128]byte, err error)

func (*SQLStore) GetBufferedEvent

func (s *SQLStore) GetBufferedEvent(ctx context.Context, ciphertextHash [32]byte) (*store.BufferedEvent, error)

func (*SQLStore) GetChatSettings

func (s *SQLStore) GetChatSettings(ctx context.Context, chat types.JID) (settings types.LocalChatSettings, err error)

func (*SQLStore) GetContact

func (s *SQLStore) GetContact(ctx context.Context, user types.JID) (types.ContactInfo, error)

func (*SQLStore) GetLatestAppStateSyncKeyID

func (s *SQLStore) GetLatestAppStateSyncKeyID(ctx context.Context) ([]byte, error)

func (*SQLStore) GetManyIdentities

func (s *SQLStore) GetManyIdentities(ctx context.Context, addresses []string) (map[string][32]byte, uint64, error)

func (*SQLStore) GetManySessions

func (s *SQLStore) GetManySessions(ctx context.Context, addresses []string) (map[string][]byte, error)

func (*SQLStore) GetMessageSecret

func (s *SQLStore) GetMessageSecret(ctx context.Context, chat, sender types.JID, id types.MessageID) (secret []byte, realSender types.JID, err error)

func (*SQLStore) GetNCTSalt

func (s *SQLStore) GetNCTSalt(ctx context.Context) ([]byte, error)

func (*SQLStore) GetOrGenPreKeys

func (s *SQLStore) GetOrGenPreKeys(ctx context.Context, count uint32) ([]*keys.PreKey, error)

func (*SQLStore) GetOutgoingEvent

func (s *SQLStore) GetOutgoingEvent(ctx context.Context, chatJID, altChatJID types.JID, id types.MessageID) (format string, result []byte, err error)

func (*SQLStore) GetPreKey

func (s *SQLStore) GetPreKey(ctx context.Context, id uint32) (*keys.PreKey, error)

func (*SQLStore) GetPrivacyToken

func (s *SQLStore) GetPrivacyToken(ctx context.Context, user types.JID) (*store.PrivacyToken, error)

func (*SQLStore) GetSenderKey

func (s *SQLStore) GetSenderKey(ctx context.Context, group, user string) (key []byte, err error)

func (*SQLStore) GetSession

func (s *SQLStore) GetSession(ctx context.Context, address string) (session []byte, err error)

func (*SQLStore) GetWASARootSecretID

func (s *SQLStore) GetWASARootSecretID(ctx context.Context, chat types.JID) (id types.MessageID, err error)

func (*SQLStore) HasSession

func (s *SQLStore) HasSession(ctx context.Context, address string) (has bool, err error)

func (*SQLStore) IsTrustedIdentity

func (s *SQLStore) IsTrustedIdentity(ctx context.Context, address string, key [32]byte) (bool, error)

func (*SQLStore) IterateSession

func (s *SQLStore) IterateSession(ctx context.Context, address string, callback func([]byte) error) (bool, error)

func (*SQLStore) IterateSessions

func (s *SQLStore) IterateSessions(ctx context.Context, addresses []string, callback func(string, []byte) error) error

func (*SQLStore) MarkPreKeysAsUploaded

func (s *SQLStore) MarkPreKeysAsUploaded(ctx context.Context, upToID uint32) error

func (*SQLStore) MigratePNToLID

func (s *SQLStore) MigratePNToLID(ctx context.Context, pn, lid types.JID) error

func (*SQLStore) PutAllContactNames

func (s *SQLStore) PutAllContactNames(ctx context.Context, contacts []store.ContactEntry) error

func (*SQLStore) PutAppStateMutationMACs

func (s *SQLStore) PutAppStateMutationMACs(ctx context.Context, name string, version uint64, mutations []store.AppStateMutationMAC) error

func (*SQLStore) PutAppStateSyncKey

func (s *SQLStore) PutAppStateSyncKey(ctx context.Context, id []byte, key store.AppStateSyncKey) error

func (*SQLStore) PutAppStateVersion

func (s *SQLStore) PutAppStateVersion(ctx context.Context, name string, version uint64, hash [128]byte) error

func (*SQLStore) PutArchived

func (s *SQLStore) PutArchived(ctx context.Context, chat types.JID, archived bool) error

func (*SQLStore) PutBufferedEvent

func (s *SQLStore) PutBufferedEvent(ctx context.Context, ciphertextHash [32]byte, plaintext []byte, serverTimestamp time.Time) error

func (*SQLStore) PutBusinessName

func (s *SQLStore) PutBusinessName(ctx context.Context, user types.JID, businessName string) (bool, string, error)

func (*SQLStore) PutContactName

func (s *SQLStore) PutContactName(ctx context.Context, user types.JID, firstName, fullName string) error

func (*SQLStore) PutContactUsername

func (s *SQLStore) PutContactUsername(ctx context.Context, user types.JID, username string) error

func (*SQLStore) PutIdentity

func (s *SQLStore) PutIdentity(ctx context.Context, address string, key [32]byte) error

func (*SQLStore) PutManyContactUsernames

func (s *SQLStore) PutManyContactUsernames(ctx context.Context, entries []store.ContactUsernameEntry) error

func (*SQLStore) PutManyRedactedPhones

func (s *SQLStore) PutManyRedactedPhones(ctx context.Context, entries []store.RedactedPhoneEntry) error

func (*SQLStore) PutManySessions

func (s *SQLStore) PutManySessions(ctx context.Context, sessions map[string][]byte) error

func (*SQLStore) PutMessageSecret

func (s *SQLStore) PutMessageSecret(ctx context.Context, chat, sender types.JID, id types.MessageID, secret []byte) (err error)

func (*SQLStore) PutMessageSecrets

func (s *SQLStore) PutMessageSecrets(ctx context.Context, inserts []store.MessageSecretInsert) (err error)

func (*SQLStore) PutMutedUntil

func (s *SQLStore) PutMutedUntil(ctx context.Context, chat types.JID, mutedUntil time.Time) error

func (*SQLStore) PutNCTSalt

func (s *SQLStore) PutNCTSalt(ctx context.Context, salt []byte) error

func (*SQLStore) PutPinned

func (s *SQLStore) PutPinned(ctx context.Context, chat types.JID, pinned bool) error

func (*SQLStore) PutPrivacyTokens

func (s *SQLStore) PutPrivacyTokens(ctx context.Context, tokens ...store.PrivacyToken) error

func (*SQLStore) PutPushName

func (s *SQLStore) PutPushName(ctx context.Context, user types.JID, pushName string) (bool, string, error)

func (*SQLStore) PutSenderKey

func (s *SQLStore) PutSenderKey(ctx context.Context, group, user string, session []byte) error

func (*SQLStore) PutSession

func (s *SQLStore) PutSession(ctx context.Context, address string, session []byte) error

func (*SQLStore) PutWASARootSecretID

func (s *SQLStore) PutWASARootSecretID(ctx context.Context, chat types.JID, id types.MessageID) error

func (*SQLStore) RemovePreKey

func (s *SQLStore) RemovePreKey(ctx context.Context, id uint32) error

func (*SQLStore) UploadedPreKeyCount

func (s *SQLStore) UploadedPreKeyCount(ctx context.Context) (count int, err error)

Directories

Path Synopsis
internal
database
Package database implements the private SQL execution and telemetry layer.
Package database implements the private SQL execution and telemetry layer.
schema
Package schema owns the adapter's embedded MariaDB schema and migrations.
Package schema owns the adapter's embedded MariaDB schema and migrations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL