A high-performance, persistent key-value storage library written in Go. Supports both B+tree and LSM-tree storage engines with WAL (Write-Ahead Logging) and snapshot-based persistence.

• Dual Storage Engines: Choose between B+tree or LSM-tree based on your workload
• Persistence: WAL + Snapshot for durability and fast recovery
• Concurrent Access: Thread-safe with fine-grained locking
• Range Queries: Efficient range scans and prefix searches
• Batch Operations: Atomic batch writes
• Configurable: Tunable parameters for different workloads
• Compression: Optional gzip compression for snapshots

go get cnb.cool/asphodelus_dev/gokv2

package main

import (
    "fmt"
    "log"

    "cnb.cool/asphodelus_dev/gokv2"
)

func main() {
    // Open database with default options (B+tree engine)
    db, err := gokv2.Open("./data")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // Put
    err = db.Put([]byte("hello"), []byte("world"))
    if err != nil {
        log.Fatal(err)
    }

    // Get
    value, err := db.Get([]byte("hello"))
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Value: %s\n", value)

    // Delete
    err = db.Delete([]byte("hello"))
    if err != nil {
        log.Fatal(err)
    }
}

// Use B+tree engine (default, optimized for read-heavy workloads)
db, err := gokv2.Open("./data", gokv2.WithStorageEngine(gokv2.BTreeEngine))

// Use LSM-tree engine (optimized for write-heavy workloads)
db, err := gokv2.Open("./data", gokv2.WithStorageEngine(gokv2.LSMEngine))

Feature	B+tree	LSM-tree
Read Performance	Excellent	Good
Write Performance	Good	Excellent
Space Amplification	Low	Medium
Write Amplification	Medium	Low
Range Scans	Excellent	Good
Best For	Read-heavy, OLTP	Write-heavy, Logs

// Open database
func Open(path string, opts ...Option) (*DB, error)

// Close database
func (db *DB) Close() error

// Get value by key
func (db *DB) Get(key []byte) ([]byte, error)

// Put key-value pair
func (db *DB) Put(key, value []byte) error

// Delete key
func (db *DB) Delete(key []byte) error

// Check if key exists
func (db *DB) Has(key []byte) (bool, error)

// Get database size (number of keys)
func (db *DB) Size() int

// Force sync to disk
func (db *DB) Sync() error

// Create snapshot manually
func (db *DB) Snapshot() error

// Create iterator over all keys
func (db *DB) NewIterator() (*Iterator, error)

// Range iterator [start, end)
func (db *DB) Range(start, end []byte) (*Iterator, error)

// Prefix scan
func (db *DB) Scan(prefix []byte) (*Iterator, error)

// ForEach iteration
func (db *DB) ForEach(fn func(key, value []byte) bool) error

// Check if iterator is valid
func (it *Iterator) Valid() bool

// Move to next entry
func (it *Iterator) Next() bool

// Move to previous entry
func (it *Iterator) Prev() bool

// Get current key
func (it *Iterator) Key() []byte

// Get current value
func (it *Iterator) Value() []byte

// Seek to key >= target
func (it *Iterator) Seek(target []byte) bool

// Seek to first key
func (it *Iterator) SeekToFirst() bool

// Seek to last key
func (it *Iterator) SeekToLast() bool

// Close iterator
func (it *Iterator) Close() error

// Create batch
batch := db.Batch()

// Add operations
batch.Put([]byte("key1"), []byte("value1"))
batch.Put([]byte("key2"), []byte("value2"))
batch.Delete([]byte("key3"))

// Commit atomically
err := batch.Commit()

// Storage engine selection
gokv2.WithStorageEngine(gokv2.BTreeEngine)  // or gokv2.LSMEngine

// B+tree order (default: 128)
gokv2.WithBTreeOrder(256)

// WAL sync mode
gokv2.WithSyncMode(gokv2.SyncAlways)  // SyncNone, SyncBatch, SyncAlways

// Snapshot settings
gokv2.WithSnapshotInterval(time.Hour)
gokv2.WithSnapshotWALThreshold(256 * 1024 * 1024)
gokv2.WithSnapshotOpsThreshold(100000)

// Compression
gokv2.WithCompression(true)

// Read-only mode
gokv2.WithReadOnly(true)

// LSM-specific options
gokv2.WithMemTableSize(64 * 1024 * 1024)  // 64MB
gokv2.WithLevel0FileNum(4)
gokv2.WithLevelSizeMultiplier(10)

gokv2/
├── go.mod
├── gokv.go                 # Main entry, public API
├── options.go              # Configuration options
├── errors.go               # Error definitions
├── iterator.go             # Iterator interface
├── batch.go                # Batch operations
│
├── internal/
│   ├── btree/              # B+tree storage engine
│   │   ├── node.go         # Node definition
│   │   ├── tree.go         # B+tree operations
│   │   └── iterator.go     # Tree iterator
│   │
│   ├── lsm/                # LSM-tree storage engine
│   │   ├── memtable.go     # In-memory table (skiplist)
│   │   ├── sstable.go      # Sorted String Table
│   │   ├── level.go        # Level management
│   │   ├── compaction.go   # Compaction strategy
│   │   └── lsm.go          # LSM-tree manager
│   │
│   ├── wal/                # Write-Ahead Log
│   │   ├── wal.go          # WAL manager
│   │   ├── entry.go        # Log entry format
│   │   ├── segment.go      # Segment management
│   │   └── reader.go       # Log reader
│   │
│   ├── snapshot/           # Snapshot management
│   │   ├── snapshot.go     # Snapshot manager
│   │   └── format.go       # Snapshot format
│   │
│   └── engine/             # Storage engine abstraction
│       ├── engine.go       # Engine interface
│       ├── btree_engine.go # B+tree engine implementation
│       ├── lsm_engine.go   # LSM engine implementation
│       ├── recovery.go     # Data recovery
│       └── compaction.go   # Compaction triggers
│
└── cmd/
    └── test/
        └── main.go         # Integration test

Write Path:
  Client -> DB.Put() -> WAL.Write() -> Engine.Put() -> [B+tree/LSM] -> Memory

Read Path:
  Client -> DB.Get() -> Engine.Get() -> [B+tree/LSM] -> Memory/Disk

Recovery Path:
  Startup -> Load Snapshot -> Replay WAL -> Ready

| CRC32 (4B) | Type (1B) | KeyLen (4B) | ValueLen (4B) | Timestamp (8B) | SeqNum (8B) | Key | Value |

| Header (32B) | [Compressed Data] | Checksum (4B) |

Header:
| Magic (4B) | Version (4B) | SeqNum (8B) | EntryCount (8B) | Flags (4B) | Reserved (4B) |

Benchmark results on typical hardware (Intel i7, NVMe SSD):

Operation	B+tree	LSM-tree
Random Write	~150K ops/s	~300K ops/s
Random Read	~500K ops/s	~200K ops/s
Range Scan (1000 keys)	~2M keys/s	~1M keys/s
Batch Write (100 ops)	~800K ops/s	~1.5M ops/s

var (
    ErrKeyNotFound     // Key does not exist
    ErrKeyEmpty        // Key cannot be empty
    ErrKeyTooLarge     // Key exceeds max size
    ErrValueTooLarge   // Value exceeds max size
    ErrDatabaseClosed  // Database is closed
    ErrCorruptedData   // Data corruption detected
    ErrBatchClosed     // Batch already committed/closed
    ErrIteratorClosed  // Iterator is closed
)

All public methods are thread-safe. The library uses:

• RWMutex for read/write separation
• Fine-grained locking in B+tree nodes
• Lock-free structures where possible in LSM-tree

MIT License