Scale Email-Sending Platform with Laravel & ClickHouse

Title: Senior engineer — scale a Laravel + ClickHouse email-sending platform to 1M+/day Description: We run a Laravel/PHP email-sending platform (MySQL, ClickHouse) that delivers through ESP/SMTP relays (Amazon SES, SendGrid, Mailgun, SparkPost). It already has a full sending control plane — per-server warmup quotas, a 3-layer volume throttle, reservation accounting, provider cooldowns, SMTP classification, MX classification. It works today at low volume. We need a senior engineer to re-architect the hot paths so it reliably scales from current volume to 100k → 250k → 500k → ~1,000,000+ messages/day without violating per-domain / per-provider / warmup limits. Concrete work areas: Rate-accounting & distributed concurrency — we have both a file-based and a Redis-cache-reservation path; benchmark contention/correctness under multiple worker hosts and move hot counting to atomic, shardable accounting. Recipient selection — replace a growing per-wave exclusion query with an indexed dispatch-ledger/cursor design. Dispatch throughput — replace serial ~200-recipient waves with a design that keeps workers saturated without flooding the queue. ClickHouse ingestion — replace one-insert-per-event with buffered/bulk batch inserts (this is a first-class requirement). Queue backend + multi-host worker topology, MySQL indexing/EXPLAIN at 30M+ rows, load testing and observability. Required (must have all): Strong PHP/Laravel (queues, jobs, Eloquent) Distributed systems / high concurrency: correct concurrent rate limiting, distributed locks, atomic counters (Redis/Lua), reservation & idempotency Laravel queue/worker scaling (database vs Redis vs SQS), Supervisor, multi-host fleets MySQL at scale: indexing, EXPLAIN, hot-row contention, tens of millions of rows ClickHouse in production — MergeTree schema, batched/bulk ingestion, avoiding part-per-insert (must have operated it, not just queried it) Proven experience operating a 500k–1M+/day email-sending system (this is the key differentiator) Load testing + production observability Nice to have: PowerMTA/Halon/MailerQ (only relevant if we later run our own MTA — not required now); deliverability fundamentals; ex-ESP engineer (SendGrid/Twilio, Mailgun, SES, SparkPost). How this starts: a paid ~20–40 hour architecture + benchmark review first — assess the production system, benchmark the bottlenecks, and deliver a target architecture + migration plan + implementation estimate. Implementation is a separate, larger engagement awarded after that review. To bid, you MUST answer these two questions (bids that don't will be ignored): How would you batch our ClickHouse event ingestion (sends/opens/clicks/bounces) to avoid part-per-insert at ~1M events/day, and how do you handle failures/duplicates? How would you design a distributed rate limiter so 20 workers across 3 hosts never exceed a per-provider hourly cap, with correct behavior if a worker dies mid-send? Please link a specific project where you scaled email sending or high-throughput ClickHouse ingestion. Do not bid if your approach relies on rotating domains/IPs or "guaranteed inbox" tactics — we protect long-term sender reputation.