golog

🔪 Golog

A high-performance, structured logging library for Go services. Golog provides standardized logging with automatic log rotation, context-aware tracing, and sensitive data masking.

Features

• 🚀 High Performance: Asynchronous buffered I/O for optimal performance
• 📝 Structured Logging: JSON-formatted logs with context-aware fields
• 🔄 Automatic Rotation: Configurable log file rotation based on size and age
• 🔒 Security: Automatic masking of sensitive data (passwords, tokens, etc.)
• 📊 TDR Logging: Transaction Detail Request logging for API requests/responses
• 🎯 Type-Safe Context: Typed context keys for better code safety
• 🔌 Flexible Usage: Singleton pattern or direct logger instances
• 📦 Production Ready: Built on top of zap logger

Installation

Install the library using Go modules:

go get -u github.com/tommynurwantoro/golog

Quick Start

package main

import (
    "context"
    "github.com/tommynurwantoro/golog"
)

func main() {
    // Initialize logger
    config := golog.Config{
        App:          "myapp",
        AppVer:       "1.0.0",
        Env:          "development",
        FileLocation: "/var/log/myapp",
        FileMaxSize:  100, // megabytes
        FileMaxBackup: 5,
        FileMaxAge:   30, // days
        Stdout:       true,
    }

    logger := golog.Load(config)
    defer golog.Sync()

    // Create context with trace information
    ctx := context.Background()
    ctx = golog.WithTraceID(ctx, "trace-123")

    // Log messages (bind context via WithContext)
    logger.WithContext(ctx).Info("Application started")
    logger.WithContext(ctx).Debug("Debug information")
    logger.WithContext(ctx).Warn("Warning message")
}

Configuration

The Config struct allows you to customize logging behavior. All fields are validated and defaults are applied automatically.

Configuration Fields

Example Configuration

config := golog.Config{
    App:            "myapp",
    AppVer:         "1.0.0",
    Env:            "production",
    FileLocation:   "/var/log/myapp",
    FileTDRLocation: "/var/log/myapp", // Optional (defaults to FileLocation)
    FileMaxSize:    500,                        // 500 MB
    FileMaxBackup:  10,                         // Keep 10 backups
    FileMaxAge:     90,                         // Keep for 90 days
    Stdout:         false,                      // Disable in production
    LogLevel:       zapcore.InfoLevel,         // Optional
    VersionFilePath: "version.txt",             // Optional
}

Usage

Singleton Pattern (Recommended)

The singleton pattern allows you to use the logger from anywhere in your application without passing it around:

package main

import (
    "context"
    "github.com/tommynurwantoro/golog"
    "go.uber.org/zap"
)

func main() {
    config := golog.Config{
        App:          "myapp",
        AppVer:       "1.0.0",
        Env:          "development",
        FileLocation: "/var/log/myapp",
        FileMaxSize:  100,
        FileMaxBackup: 5,
        FileMaxAge:   30,
        Stdout:       true,
    }

    // Initialize singleton logger
    golog.Load(config)
    defer golog.Sync() // Important: flush logs on exit

    // Use from anywhere - bind context via WithContext, then call log methods
    ctx := context.Background()
    ctx = golog.WithTraceID(ctx, "trace-123")

    golog.WithContext(ctx).Info("Application started")
    golog.WithContext(ctx).Debug("Debug message", zap.String("key", "value"))
    golog.WithContext(ctx).Warn("Warning message")
}

Direct Logger Instance

For more control or when you need multiple logger instances:

package main

import (
    "context"
    "github.com/tommynurwantoro/golog"
)

func main() {
    config := golog.Config{
        App:          "myapp",
        AppVer:       "1.0.0",
        Env:          "production",
        FileLocation: "/var/log/myapp",
        FileMaxSize:  500,
        FileMaxBackup: 10,
        FileMaxAge:   90,
        Stdout:       false,
    }

    logger := golog.NewLogger(config)
    defer logger.Sync() // Important: flush logs before exit

    ctx := context.Background()
    ctx = golog.WithTraceID(ctx, "trace-123")
    logger.WithContext(ctx).Info("Application started")
}

Logging Methods

Bind context via WithContext(ctx), then call logging methods with message and optional fields:

ctx := context.Background()
ctx = golog.WithTraceID(ctx, "trace-123")
log := golog.WithContext(ctx)

// Info level logging
log.Info("User logged in", zap.String("userID", "123"))

// Debug level logging
log.Debug("Processing request", zap.String("method", "GET"))

// Warning level logging
log.Warn("Rate limit approaching", zap.Int("requests", 95))

// Error level logging
err := errors.New("database connection failed")
log.Error("Failed to connect", err, zap.String("host", "db.example.com"))

// Fatal level logging (exits application)
// log.Fatal("Critical error", err)

// Panic level logging (panics)
// log.Panic("Unexpected error", err)

Context-Aware Logging

Golog automatically extracts trace information from the context. This makes it easy to track requests across your application.

Using Typed Context Keys (Recommended)

Type-safe context keys prevent typos and provide better IDE support:

import (
    "context"
    "github.com/tommynurwantoro/golog"
)

func handleRequest(ctx context.Context) {
    // Add trace information to context
    ctx = golog.WithTraceID(ctx, "trace-123")
    ctx = golog.WithSrcIP(ctx, "192.168.1.1")
    ctx = golog.WithPort(ctx, "8080")
    ctx = golog.WithPath(ctx, "/api/users")

    // All logs will automatically include these fields
    golog.WithContext(ctx).Info("Request received")
    
    // Retrieve values if needed
    traceID, ok := golog.GetTraceID(ctx)
    if ok {
        // Use traceID
    }
}

Using String Keys (Backward Compatible)

For backward compatibility, you can still use string keys:

ctx = context.WithValue(ctx, "traceId", "trace-123")
ctx = context.WithValue(ctx, "srcIP", "192.168.1.1")
ctx = context.WithValue(ctx, "port", "8080")
ctx = context.WithValue(ctx, "path", "/api/users")

Available Context Keys

• traceId / golog.WithTraceID() - Request trace ID
• srcIP / golog.WithSrcIP() - Source IP address
• port / golog.WithPort() - Port number
• path / golog.WithPath() - Request path

All context values are automatically included in log entries.

Transaction Detail Request (TDR) Logging

TDR logging captures complete request/response information for API calls, including headers, request/response bodies, status codes, and response times. Sensitive data is automatically masked.

import (
    "context"
    "time"
    "github.com/tommynurwantoro/golog"
)

func logAPIRequest(ctx context.Context, req, resp interface{}) {
    tdr := golog.LogModel{
        CorrelationID: "corr-456",
        Method:        "POST",
        Path:          "/api/users",
        StatusCode:    "200",
        HttpStatus:    200,
        Header:        requestHeaders,      // http.Header or fasthttp.RequestHeader
        Request:       req,                 // Request body (will be masked)
        Response:      resp,                // Response body (will be masked)
        ResponseTime:  150 * time.Millisecond,
        Error:         nil,                 // Error if any
        OtherData:     map[string]interface{}{"custom": "data"},
    }

    golog.WithContext(ctx).TDR(tdr)
}

Note: TDR logs are written to a separate file (FileTDRLocation) for easier analysis and monitoring.

Sensitive Data Masking

Golog automatically masks sensitive fields in request/response bodies:

• password
• license, license_code
• token, access_token, refresh_token

Sensitive headers are also removed:

• Authorization
• Signature
• Apikey

Masked values are replaced with ***** in logs.

Log Output

File Output

By default, logs are written to files in the FileLocation directory:

• System logs: {FileLocation}/system.log
• TDR logs: {FileTDRLocation}/tdr.log (defaults to {FileLocation}/tdr.log)

Console Output

Enable console output for development by setting Stdout: true:

config := golog.Config{
    // ... other config
    Stdout: true, // Logs will appear in console
}

In production, set Stdout: false for better performance.

Log Rotation

Log files are automatically rotated when they reach FileMaxSize:

• Old files are renamed with a timestamp suffix
• Only FileMaxBackup backup files are kept
• Files older than FileMaxAge days are automatically deleted

Example rotation:

system.log
system.log.2024-01-15
system.log.2024-01-14

Log Format

Logs are written in JSON format for easy parsing:

{
  "timestamp": "2024-01-15T10:30:45Z",
  "logLevel": "INFO",
  "message": "User logged in",
  "app": "myapp",
  "appVer": "1.0.0",
  "env": "production",
  "traceId": "trace-123",
  "srcIP": "192.168.1.1",
  "path": "/api/users",
  "userID": "123"
}

Performance Considerations

Best Practices

1. Always call Sync() on shutdown: Ensures all buffered logs are written

   defer golog.Sync() // For singleton
   defer logger.Sync() // For direct logger
   

2. Use appropriate log levels: Set LogLevel to filter out unnecessary logs in production

   config.LogLevel = zapcore.InfoLevel // Only log Info and above
   

3. Disable stdout in production: Console output adds overhead

   config.Stdout = false // Production
   config.Stdout = true  // Development
   

4. Use context efficiently: Pre-populate context at request entry point and bind once

   // At HTTP handler entry
   ctx = golog.WithTraceID(ctx, generateTraceID())
   ctx = golog.WithPath(ctx, r.URL.Path)
   log := golog.WithContext(ctx)
   // Use log.Info(), log.Error(), etc. throughout the handler
   

Performance Characteristics

• Asynchronous I/O: Logs are buffered and written asynchronously
• Zero-allocation context extraction: Context fields are extracted efficiently
• Optimized JSON operations: Sensitive data masking is optimized for performance

Advanced Usage

Custom Log Levels

import "go.uber.org/zap/zapcore"

config := golog.Config{
    // ... other config
    LogLevel: zapcore.DebugLevel, // Enable debug logs
}

Version File Override

If a version file exists, it will override AppVer:

config := golog.Config{
    AppVer:         "1.0.0",        // Default version
    VersionFilePath: "version.txt", // File containing "1.2.3"
    // AppVer will be "1.2.3" if file exists
}

Testing

Reset the singleton logger between tests:

func TestSomething(t *testing.T) {
    golog.Reset() // Reset singleton
    
    config := golog.Config{/* test config */}
    logger := golog.Load(config)
    defer golog.Sync()
    
    // Your test code
}

Troubleshooting

Logs not appearing

• Ensure Sync() is called before application exit
• Check file permissions for FileLocation directory
• Verify LogLevel allows the log level you're using

Performance issues

• Disable Stdout in production
• Increase FileMaxSize to reduce rotation frequency
• Use appropriate LogLevel to filter unnecessary logs

Missing context fields

• Ensure context is passed through your call chain
• Bind context before logging: golog.WithContext(ctx).Info(...) or logger.WithContext(ctx).Info(...)
• Use typed context keys (golog.WithTraceID()) for type safety
• Check that context values are set before logging

Examples

See the logger_test.go file for usage examples including:

• Basic logging
• Context binding with WithContext
• Error handling
• TDR logging
• Configuration and validation

Contributing

Contributions are welcome! Please follow the Contribution Guidelines.

Development Setup

git clone https://github.com/tommynurwantoro/golog.git
cd golog
go mod download
go test ./...

License

This project is licensed under the MIT License. See the LICENSE file for details.

Acknowledgments

• Built on top of zap - A blazing fast, structured, leveled logging library
• Uses lumberjack for log rotation