# MindooDB

> End-to-end encrypted, offline-first sync database. Keys stay on devices; servers store and sync ciphertext. Open source (Apache 2.0). For Node.js, browsers, and React Native.

MindooDB is an open-source database for building secure, collaborative applications where encryption keys never leave client devices. Data is encrypted before it touches a server, signed for authorship proof, and stored in an append-only, cryptographically chained history. Servers can store and relay data but cannot read it — even a complete server breach yields only ciphertext and public keys.

The database is built on Automerge CRDTs for automatic conflict-free merging, a content-addressed store abstraction for flexible deployment topologies (client-server, peer-to-peer, relay, mesh), and a metadata-first sync protocol that transfers only missing entries. It supports chunked encrypted attachments, hierarchical Virtual Views (inspired by HCL Notes/Domino), incremental cursor-based indexing, and time travel queries over the full document history.

MindooDB is alpha software. APIs may change without notice.

## Core Concepts

### Tenants
A tenant represents an organization or team. Created entirely client-side with no server registration. The tenant creator becomes the administrator whose Ed25519 signing key is the root of trust. Each tenant contains a directory database (admin-signed user registry), multiple application databases, and encryption keys that control access.

### Users
Identified by cryptographic key pairs: an Ed25519 signing key (proves authorship) and an RSA-OAEP encryption key (protects local key storage). Both private keys are encrypted with a single password using PBKDF2 with different salts. New users join through a three-step handshake (join request / admin approval / join response) where private keys never leave the originating device.

### Documents
Each document is an Automerge CRDT stored in a content-addressed store. Every change is signed (Ed25519) and encrypted (AES-256-GCM) before storage. Complete history is preserved in an append-only chain. Documents support time travel (reconstruct any historical state) and periodic snapshots for performance.

### Content-Addressed Store
The unified storage and sync abstraction. Entries are identified by structured IDs and deduplicated by content hash (SHA-256 of encrypted payload). The same interface backs local disk, in-memory, and remote network stores — making sync composable across any topology. Core operations: putEntries, getEntries, findNewEntries, resolveDependencies.

### Encryption Model
Three-layer defense in depth: (1) AES-256-GCM application-level encryption at rest, (2) per-user RSA-OAEP transport encryption, (3) TLS channel encryption. Access control is enforced through encryption keys — if you have the key, you can decrypt; if not, the document is ciphertext. The default tenant key is shared with all members; named keys provide fine-grained access for specific users or groups.

### KeyBag
Local password-protected storage for symmetric keys. Encrypted on disk using the user's encryption key via PBKDF2. Stores both the tenant default key and any named document keys. Supports key import/export for distribution.

### Sync Protocol
Metadata-first reconciliation: exchange entry IDs to find what's missing, then transfer only missing entries. Supports capability negotiation for optimizations: cursor-based scanning (constant request size), Bloom filter summaries (eliminate 90-99% of existence checks), and CRDT snapshots. Authentication uses Ed25519 challenge-response with short-lived JWTs.

### Attachments
Files attached to documents: chunked (256KB default), encrypted with the same key as the parent document, with deterministic IV for tenant-wide deduplication. Supports streaming upload/download for large files. Stored in a separate content-addressed store for flexible deployment.

### Virtual Views
Hierarchical in-memory views that categorize, sort, and aggregate documents — like a dynamic table of contents. Support nested category hierarchies, multi-column sorting, SUM/AVERAGE aggregations, cross-database and cross-tenant data sources, and incremental updates. Inspired by HCL Notes/Domino views.

### Time Travel
Retrieve document state at any historical timestamp with getDocumentAtTimestamp(). Traverse complete change history with iterateDocumentHistory() including author metadata. List all document IDs at a specific time with getAllDocumentIdsAtTimestamp(). Supports compliance audits, version comparison, and point-in-time database snapshots.

### Incremental Indexing
iterateChangesSince(cursor) is the single foundation for all querying. Cursor-based async generator yields only documents that changed since the last run. Powers Virtual Views, fulltext search integration (FlexSearch, MiniSearch, Lunr.js), external system sync, and custom analytics pipelines. Cost proportional to rate of change, not total data size.

## API Quick Reference

### Create a Tenant
```typescript
import { BaseMindooTenantFactory, InMemoryContentAddressedStoreFactory } from "mindoodb";

const factory = new BaseMindooTenantFactory(new InMemoryContentAddressedStoreFactory());
const { tenant, adminUser, appUser, keyBag } = await factory.createTenant({
  tenantId: "acme",
  adminName: "cn=admin/o=acme",
  adminPassword: "admin-pw",
  userName: "cn=alice/o=acme",
  userPassword: "alice-pw",
});
```

### Create and Modify Documents
```typescript
const db = await tenant.openDB("todos");
const doc = await db.createDocument();
await db.changeDoc(doc, (d) => {
  d.getData().title = "Buy milk";
  d.getData().done = false;
});
const loaded = await db.getDocument(doc.getId());
console.log(loaded.getData()); // { title: "Buy milk", done: false }
```

### Sync with a Server
```typescript
await tenant.publishToServer("https://sync.example.com");
const remote = await tenant.connectToServer("https://sync.example.com", "todos");
await db.pushChangesTo(remote);
await db.pullChangesFrom(remote);
await db.syncStoreChanges();
```

### Invite a User (Join Request / Response)
```typescript
// New user creates identity and join request
const bob = await factory.createUserId("cn=bob/o=acme", "bob-pw");
const joinRequest = factory.createJoinRequest(bob, { format: "uri" });

// Admin approves and creates join response
const joinResponse = await tenant.approveJoinRequest(joinRequest, {
  adminSigningKey: adminUser.userSigningKeyPair.privateKey,
  adminPassword: "admin-pw",
  sharePassword: "one-time-secret",
  format: "uri",
});

// New user joins the tenant
const { tenant: bobTenant } = await factory.joinTenant(joinResponse, {
  user: bob,
  password: "bob-pw",
  sharePassword: "one-time-secret",
});
```

### Virtual Views
```typescript
const view = await VirtualViewFactory.createView()
  .addCategoryColumn("department")
  .addSortedColumn("lastName")
  .addTotalColumn("salary", TotalMode.SUM)
  .withDB("employees", employeeDB, (doc) => doc.getData().type === "employee")
  .buildAndUpdate();

const nav = VirtualViewFactory.createNavigator(view).expandAll().build();
for await (const entry of nav.entriesForward()) { /* categories + documents */ }
```

### Incremental Indexing
```typescript
let cursor = null;
for await (const { doc, cursor: newCursor } of db.iterateChangesSince(cursor)) {
  if (doc.isDeleted()) myIndex.remove(doc.getId());
  else myIndex.update(doc);
  cursor = newCursor;
}
```

### Attachments
```typescript
await db.changeDoc(doc, async (d) => {
  await d.addAttachment(fileData, "report.pdf", "application/pdf");
  await d.addAttachmentStream(asyncIterable, "video.mp4", "video/mp4");
});
const data = await doc.getAttachment(attachmentId);
for await (const chunk of doc.streamAttachment(attachmentId)) { /* streaming */ }
```

### Time Travel
```typescript
const docYesterday = await db.getDocumentAtTimestamp(docId, yesterday);
for await (const { doc, changeCreatedAt, changeCreatedByPublicKey }
  of db.iterateDocumentHistory(docId)) { /* full history */ }
const idsAtDate = await db.getAllDocumentIdsAtTimestamp(endOfQ1);
```

## Supported Platforms

| Platform | Import |
|----------|--------|
| Node.js | `import { BaseMindooTenantFactory, ... } from "mindoodb"` |
| Web Browser | `import { BaseMindooTenantFactory, ... } from "mindoodb/browser"` |
| React Native | `import { BaseMindooTenantFactory, ... } from "mindoodb"` + native Automerge via `react-native-automerge-generated` |

## Website Pages

- [Home](https://mindoodb.com/): Landing page with feature overview, quick start code examples, comparison table, deployment topologies, and role-based navigation links.
- [How It Works](https://mindoodb.com/how-it-works): Step-by-step walkthrough of multi-user setup — tenant creation, server publishing, join request/response handshake, and encrypted collaboration. Shows what the server sees vs. never sees.
- [Architecture](https://mindoodb.com/architecture): Content-addressed store design, sync protocol, deployment topologies (client-server, P2P, relay, mesh), durability and crash safety, data modeling strategies, and honest tradeoffs.
- [Security](https://mindoodb.com/security): Three-layer encryption model, trust chain (admin key to directory to users), access control enforcement, challenge-response authentication, cryptographic primitives, threat model, and metadata exposure.
- [Get Started](https://mindoodb.com/get-started): Platform picker (Node.js, Web, React Native) with quickstart code and cross-platform sync diagram.
- [Get Started: Node.js](https://mindoodb.com/get-started/node): Complete Node.js quickstart — install, create tenant, create document, read back, with expected console output.
- [Get Started: Web](https://mindoodb.com/get-started/web): Browser quickstart using mindoodb/browser with Web Crypto adapter (createCryptoAdapter).
- [Get Started: React Native](https://mindoodb.com/get-started/react-native): Native-first setup with react-native-automerge-generated, polyfills, react-native-quick-crypto, and Expo compatibility.
- [Documentation](https://mindoodb.com/docs): Curated entry points to GitHub documentation — architecture spec, sync protocol, P2P sync, attachments, Virtual Views, time travel, indexing, logging, and security.
- [Use Cases](https://mindoodb.com/use-cases): Six scenarios (offline-first field apps, secure collaboration, multi-tenant SaaS, audit/compliance, encrypted files, cross-org reporting) with key features and watch-outs. Decision framework for when to use MindooDB vs. alternatives.
- [Industries](https://mindoodb.com/industries): Healthcare (HIPAA, EHR, multi-institutional research), Financial Services (SOX, PCI-DSS, transaction ledgers), IoT & Edge Computing (sensor data, device management), Collaborative Workspaces (document management, project management, knowledge bases).
- [Compliance](https://mindoodb.com/compliance): How MindooDB supports HIPAA, SOX, GDPR, PCI-DSS through end-to-end encryption, signed append-only audit trails, fine-grained access control, and coordinated data erasure. Compliance feature matrix.
- [Indexing & Querying](https://mindoodb.com/indexing): iterateChangesSince() foundation, Virtual Views with categories/sorting/totals, cross-database and cross-tenant views, external indexer integration, time travel queries.
- [Compare](https://mindoodb.com/compare): Feature-by-feature comparison vs. PostgreSQL/MySQL, Firebase/Supabase, MongoDB, and blockchains. Migration considerations.
- [Imprint](https://mindoodb.com/imprint): Legal notice for Mindoo GmbH, Karlsruhe, Germany.

## GitHub Documentation

- [Project README](https://github.com/klehmann/MindooDB/blob/main/README.md): Quick start, core concepts, code examples, and platform support overview.
- [Architecture Specification](https://github.com/klehmann/MindooDB/blob/main/docs/specification.md): Full technical specification — tenants, users, documents, content-addressed store API, encryption model, security model, data flows, attachments, and performance.
- [Getting Started Guide](https://github.com/klehmann/MindooDB/blob/main/docs/getting-started.md): Complete setup walkthrough from tenant creation to multi-user collaboration, with session persistence.
- [Network Sync Protocol](https://github.com/klehmann/MindooDB/blob/main/docs/network-sync-protocol.md): Endpoint contracts, authentication flow, capability negotiation, Bloom filter optimization, and security model.
- [P2P Sync](https://github.com/klehmann/MindooDB/blob/main/docs/p2psync.md): Peer-to-peer communication, multi-hop relays, sync without decryption, and network composition patterns.
- [Attachments](https://github.com/klehmann/MindooDB/blob/main/docs/attachments.md): Chunked encrypted file storage, streaming upload/download, deduplication, and sync.
- [Virtual Views](https://github.com/klehmann/MindooDB/blob/main/docs/virtualview.md): Hierarchical categorized views with sorting, totals, cross-database sources, and navigation API.
- [Data Indexing](https://github.com/klehmann/MindooDB/blob/main/docs/dataindexing.md): iterateChangesSince() API, indexing strategies, and external search integration.
- [Time Travel](https://github.com/klehmann/MindooDB/blob/main/docs/timetravel.md): Historical document retrieval, history traversal, and point-in-time snapshots.
- [React Native Guide](https://github.com/klehmann/MindooDB/blob/main/docs/reactnative.md): Native Automerge setup, polyfills, Expo compatibility, and troubleshooting.
- [On-Disk Store](https://github.com/klehmann/MindooDB/blob/main/docs/on-disk-content-addressed-store.md): Node.js persistent store implementation — on-disk layout, write protocol, crash recovery, and compaction.
- [Browser IndexedDB Store](https://github.com/klehmann/MindooDB/blob/main/docs/browser-indexeddb-store.md): Browser storage using IndexedDB — schema, multi-tenant isolation, and PWA considerations.
- [Logging](https://github.com/klehmann/MindooDB/blob/main/docs/logging.md): Structured logging with automatic sanitization and hierarchical context.
- [Revocation Protection](https://github.com/klehmann/MindooDB/blob/main/docs/revocation-timestamp-protection.md): Preventing revoked users from creating backdated changes.
- [Compliance Patterns](https://github.com/klehmann/MindooDB/blob/main/docs/usecases/compliance-patterns.md): GDPR, HIPAA, SOX compliance implementation patterns.
- [Migration Patterns](https://github.com/klehmann/MindooDB/blob/main/docs/usecases/migration-patterns.md): Migrating from SQL databases, NoSQL, and cloud services.

## Source Code

- [GitHub Repository](https://github.com/klehmann/MindooDB): Full source code, issues, and discussions.
- [npm Package](https://www.npmjs.com/package/mindoodb): `npm install mindoodb`
- License: Apache 2.0
- Author: Mindoo GmbH, Karlsruhe, Germany