FileMailer-PG Module #

The FileMailer platform — now sized for the enterprise. Millions of emails, petabytes of attachments, zero duplicates, one clean API.

An enterprise-grade Spring Boot service that synchronises IMAP mailboxes into a PostgreSQL database and stores every .eml or .emlx and attachment in a S3-compatible database using content-addressable storage with SHA-256 deduplication. A full REST API exposes accounts, folders, emails, drafts, queues and admin operations — ready to integrate with FileMaker, CRM/ERP systems, or any backend that speaks HTTP.

Overview #

FileMailer-PG is designed for large and critical deployments where mailbox volume, archiving compliance, storage efficiency and uptime all matter. It decouples metadata (PostgreSQL) from physical files (Minio / S3), so each layer can scale independently.

Every incoming message is parsed, the original .eml/.emlx is preserved in Minio, attachments are extracted and stored by hash, and a rich metadata row goes into PostgreSQL — ready for full-text search, threading, and API consumption.

Key Features #

IMAP Synchronization #

• UID-based incremental sync — only new messages are fetched each cycle
• IMAP IDLE push notifications for real-time delivery, one persistent connection per account
• Distributed sync locks prevent concurrent synchronisation of the same account from IDLE, scheduler or manual triggers
• Automatic detection of UIDVALIDITY changes triggers safe folder re-sync
• Multi-folder support with configurable exclusions (Junk, Spam, Trash…) and auto-restore

Content-Addressable Storage (CAS) with Deduplication #

• Every email and attachment stored in Minio by its SHA-256 hash
• Identical content is stored only once — up to 80% storage savings on typical corporate mailboxes
• Path layout: {prefix}/{accountId}/{hash[0:2]}/{hash[2:4]}/{hash}
• Bucket versioning enabled for immutable archiving and retention
• Streaming upload for large attachments — memory-safe for multi-GB files

PostgreSQL Metadata Layer #

• Full relational schema: accounts, folders, emails, attachments, stored files, API keys
• Full-text search on subject and body content
• Email threading via In-Reply-To and References
• Soft deletes (isDeleted, deletedAt) for GDPR-compliant recovery, with optional hard-delete sweep
• Flyway-managed migrations, HikariCP connection pooling

SMTP Sending & Queue #

• Asynchronous sending via a dedicated ThreadPoolExecutor (@Async queue)
• Retry logic with exponential backoff for transient SMTP failures
• Tracking IDs per send — clients poll status, no blocking on SMTP
• Draft creation, editing, attachment handling, reply and forward
• Multiple SMTP configurations per account (SSL 465 / STARTTLS 587)
• Graceful shutdown waits for in-flight sends

Powerful REST API #

• Fully documented with Swagger UI and OpenAPI 3
• Stateless authentication via X-API-Key header — no sessions, no cookies
• Endpoints for accounts, folders, emails, attachments, drafts, queue, admin, auth
• Custom header injection for FileMaker / CRM integration workflows
• Email re-parse endpoints to rebuild metadata from the original .eml stored in Minio

Security #

• IMAP / SMTP credentials encrypted with AES-GCM before persistence — raw passwords never leave the wire
• ApiKeyAuthenticationFilter validates each request and tracks lastUsedAt
• Spring Security configured for stateless REST
• CORS configuration ready for browser clients

Scalability & Operations #

• Horizontal scaling: add PostgreSQL replicas and Minio nodes independently
• Vertical scaling: Java 17~21, Spring Boot 3.4, tuned HikariCP and executor pools
• Unlimited email accounts — no per-account licensing, no artificial caps
• Health and metric endpoints ready for monitoring stacks
• macOS launchd plist included for background service operation

Use Cases #

• Enterprise email archiving & compliance — every message preserved, immutable, deduplicated
• Mission-critical FileMaker back-office — pair with FileMaker via the REST API for unlimited mailbox scale
• CRM / ERP integration — attach customer communication history through a clean HTTP API
• Document lifecycle management — attachment deduplication across the entire organisation
• Long-term retention — Minio versioning keeps historical copies safe from overwriting

Technical Specifications #

Stack #

• Java 17
• Spring Boot 3.4.x
• PostgreSQL (primary metadata store)
• Minio / any S3-compatible object storage (physical files)
• Jakarta Mail (Eclipse Angus) for IMAP / SMTP
• Spring Security, Spring Data JPA, HikariCP
• Flyway for database migrations
• SpringDoc OpenAPI + Swagger UI

Requirements #

• Java 17 runtime
• PostgreSQL server
• Minio (or any S3-compatible endpoint) with versioning enabled
• Network access to IMAP and SMTP servers
• 1 GB RAM minimum (2 GB+ recommended for large mailboxes)

Deployment #

Distributed as an executable Spring Boot JAR with external application.yml. A macOS launchd plist is included for background service operation. The same JAR scales from a single-node VM to a multi-replica Kubernetes deployment backed by PostgreSQL HA and a clustered Minio pool.

Pricing #