FastEmbed-go

Go implementation of @Qdrant/fastembed

Fork changes and features

• install via go get github.com/bdombro/fastembed-go@latest
• Updated golang and deps
• Added support for onnx when installed via homebrew on Macos

🍕 Features

• Supports batch embeddings with parallelism using go-routines.
• Uses @sugarme/tokenizer for fast tokenization.
• Optimized embedding models.

The default embedding supports "query" and "passage" prefixes for the input text. The default model is Flag Embedding, which is top of the MTEB leaderboard.

🔍 Not looking for Go?

• Python 🐍: fastembed
• Rust 🦀: fastembed-rs
• JavaScript 🌐: fastembed-js

🤖 Models

• BAAI/bge-base-en
• BAAI/bge-base-en-v1.5
• BAAI/bge-small-en
• BAAI/bge-small-en-v1.5 - Default
• BAAI/bge-base-zh-v1.5
• sentence-transformers/all-MiniLM-L6-v2

🚀 Installation

Run the following Go CLI command in your project directory:

go get -u github.com/anush008/fastembed-go

ℹ︎ Notice:

On macOS, ONNX Runtime from Homebrew (brew install onnxruntime) is detected automatically under /opt/homebrew or /usr/local.

On Linux, the underlying onnxruntime_go loader looks for a shared library named onnxruntime.so in the process working directory by default. After unpacking the official release, symlink the real library to that name from your project root, for example:

ln -sf /path/to/onnxruntime-*/lib/libonnxruntime.so onnxruntime.so

If you still see Error loading ONNX shared library, confirm the symlink exists where you run the binary or tests, or call onnxruntime_go.SetSharedLibraryPath yourself before initializing this package.

📖 Usage

import "github.com/anush008/fastembed-go"

// With default options
model, err := fastembed.NewFlagEmbedding(nil)
if err != nil {
 panic(err)
}
defer model.Destroy()

// With custom options
options := fastembed.InitOptions{
 Model:     fastembed.BGEBaseEN,
 CacheDir:  "model_cache",
 MaxLength: 200,
}

model, err = fastembed.NewFlagEmbedding(&options)
if err != nil {
 panic(err)
}
defer model.Destroy()

documents := []string{
 "passage: Hello, World!",
 "query: Hello, World!",
 "passage: This is an example passage.",
 // You can leave out the prefix but it's recommended
 "fastembed-go is licensed under MIT",
}

// Generate embeddings with a batch-size of 25, defaults to 256
embeddings, err := model.Embed(documents, 25)  //  -> Embeddings length: 4
if err != nil {
 panic(err)
}

Supports passage and query embeddings for more accurate results

// Generate embeddings for the passages
// The texts are prefixed with "passage" for better results
// The batch size is set to 1 for demonstration purposes
passages := []string{
 "This is the first passage. It contains provides more context for retrieval.",
 "Here's the second passage, which is longer than the first one. It includes additional information.",
 "And this is the third passage, the longest of all. It contains several sentences and is meant for more extensive testing.",
}

embeddings, err := model.PassageEmbed(passages, 1)  //  -> Embeddings length: 3
if err != nil {
 panic(err)
}

// Generate embeddings for the query
// The text is prefixed with "query" for better retrieval
query := "What is the answer to this generic question?";

embeddings, err := model.QueryEmbed(query)
if err != nil {
 panic(err)
}

🚒 Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

1. Quantized model weights
2. ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Why light?

1. No hidden dependencies via Huggingface Transformers

Why accurate?

1. Better than OpenAI Ada-002
2. Top of the Embedding leaderboards e.g. MTEB

📄 LICENSE

MIT © 2023