How MerkleMap Works

MerkleMap is a powerful subdomain search engine and certificate transparency search engine that allows you to efficiently search for and discover subdomains and SSL/TLS certificates across the internet. This guide will explain how MerkleMap collects and stores Certificate Transparency data to provide comprehensive search capabilities.

Certificate Transparency

Certificate Transparency (CT) is an open framework that aims to monitor and audit SSL/TLS certificates issued by Certificate Authorities (CAs). It provides a public, append-only log of all issued certificates, allowing domain owners, security researchers, and the general public to monitor and verify the issuance and validity of certificates.

When a CA issues a new SSL/TLS certificate, it is required to submit the certificate to one or more CT logs. These logs are append-only, meaning that once a certificate is added, it cannot be removed or modified. Each log entry contains information about the issued certificate, including the domain name, issuer, validity period, and other relevant details.

Live Tailing of CT Logs

MerkleMap collects Certificate Transparency data by continuously monitoring and live tailing the CT logs. Here's how the process works:

Identifying CT Logs: MerkleMap maintains a list of known CT logs operated by various organizations, such as Google, Cloudflare, and Let's Encrypt. These logs are trusted sources of certificate data.
Establishing Connections: MerkleMap establishes persistent connections to the CT logs. These connections allow MerkleMap to receive real-time updates whenever new certificates are added to the logs.
Retrieving Certificate Data: Whenever a new certificate is submitted to a CT log, MerkleMap receives a notification through the established connection. It then retrieves the certificate data, including the domain name, issuer, validity period, and other relevant information.
Parsing and Indexing: MerkleMap parses the retrieved certificate data and extracts the necessary information. It then indexes the data based on various attributes, such as the domain name, issuer, and expiration date, to enable efficient search and retrieval.
Storing in a Database: The parsed and indexed certificate data is stored in MerkleMap's database. This database is optimized for fast querying and supports complex search operations, allowing users to search for certificates based on different criteria.

Comprehensive Data Availability

By continuously live tailing the CT logs, MerkleMap ensures that it has the most up-to-date and comprehensive collection of SSL/TLS certificate data. This data is made available to users through MerkleMap's web interface, API, and command-line interface (CLI).

Users can perform searches based on various criteria, such as:

Domain names and subdomains
Certificate issuers
Validity periods
Certificate fingerprints
And more

MerkleMap's search capabilities allow users to quickly and easily discover certificates associated with specific domains, investigate potential security issues, monitor certificate deployments, and gain insights into the SSL/TLS certificate landscape.