> ## Documentation Index
> Fetch the complete documentation index at: https://envtrap.vercel.app/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Secret Detection Engine

> How envtrap loads secrets from the environment, evaluates entropy, and matches exfiltration attempts at runtime.

# Secret Detection & Entropy Engine

`envtrap` detects secrets in a two-stage lifecycle:

1. **Startup Discovery**: Reads and evaluates secret candidates from `process.env` and `.env` files.
2. **Runtime Matching**: Performs ultra-fast exact substring matching on live egress buffers without heavy regex runtime overhead.

***

## Stage 1 — Secret Candidate Discovery

When `envtrap run` starts, it initializes an in-memory secret registry by merging two distinct sources:

<CardGroup cols={2}>
  <Card title="process.env (Active Shell)" icon="terminal">
    Evaluates every environment variable in the current shell. A built-in blocklist filters out non-sensitive system environment variables (`PATH`, `HOME`, `USER`, `SHELL`, `PWD`, `LANG`, `TERM`, `NODE_ENV`, `NODE_OPTIONS`, `XDG_*`, etc.).
  </Card>

  <Card title="File Secrets (.env)" icon="file-code">
    Parses `.env` (or custom paths provided via `--env-file`). In v3.1, secrets explicitly declared in `.env` files are strictly retained and protected even if their variable name coincides with generic environment names, ensuring targeted secrets are never discarded.
  </Card>
</CardGroup>

***

## Stage 2 — The Candidate Evaluation Gate

Every string value from the environment must pass through the candidate filter before being added to the active watch list:

<Steps>
  <Step title="Length Threshold Check">
    Strings with fewer than `minLength` characters (default: `12`) are discarded immediately. This ensures short non-secret variables (e.g. `PORT=3000`, `DEBUG=true`, `ENV=prod`) never trigger false alerts.
  </Step>

  <Step title="Deterministic Regex Pattern Matching">
    If the string matches any known vendor format (Stripe, AWS, GitHub, Slack, SendGrid, or Bearer tokens), it is **instantly registered as a secret**, completely bypassing the entropy calculation.
  </Step>

  <Step title="Shannon Entropy Analysis">
    If the string does not match a known prefix, envtrap computes its Shannon entropy. If the score is greater than or equal to `entropy.threshold` (default: `3.5` bits/char), it is registered as an active secret candidate.
  </Step>
</Steps>

***

## Built-In Deterministic Patterns

These high-priority patterns always match, regardless of entropy score:

| Provider | Pattern Description | Format Example |
| :- | :- | :- |
| **Stripe Secret Key** | `sk_live_` or `sk_test_` + 24+ alphanumeric characters | `sk_live_51Nz...` |
| **AWS Access Key ID** | `AKIA` + 16 uppercase alphanumeric characters | `AKIAIOSFODNN7EXAMPLE` |
| **GitHub PAT** | `ghp_` + 36 alphanumeric characters | `ghp_abc123...` |
| **SendGrid API Key** | `SG.` + 22 characters + `.` + 43 characters | `SG.xxx.yyy` |
| **Slack Bot Token** | `xoxb-` + numeric IDs + 24 alphanumeric characters | `xoxb-123-456-...` |
| **Generic Bearer Token** | `Bearer` + 20+ URL-safe base64/token characters | `Bearer eyJhbGciOi...` |

***

## Shannon Entropy Calculation

Shannon entropy measures the uncertainty and statistical randomness of the characters in a string:

$H = -\sum_{i=1}^{n} p_i \log_2(p_i)$

Where $p_i$ is the frequency probability of each unique character in the string. The higher the randomness and character diversity, the higher the score (0 to 8 bits/character).

| Example String | Entropy Score | Classification |
| :- | :- | :- |
| `hello world` | ≈ `2.8` | Low — standard natural language text |
| `user_john_doe` | ≈ `3.1` | Low — human-readable identifier |
| `db_prod_2026` | ≈ `3.2` | Low — drops below default threshold (`3.5`) |
| `aB3!kR9mNq2Xp7v` | ≈ `4.1` | High — flagged as an active secret |
| `ghp_xyz012345abc...` | ≈ `5.8` | High — flagged by pattern + entropy |

<Tip>
  You can tune the entropy gate in `envtrap.json`. If your project uses many semi-random config values that cause false positives, raise `entropy.threshold` to `4.0` or `4.2`.
</Tip>

***

## High-Performance Runtime Scanning

During application execution, every intercepted HTTP payload, terminal output chunk, DNS query, and subprocess environment is scanned for **exact substring matches** of the active secrets list:

<AccordionGroup>
  <Accordion title="O(1) Direct Inclusion vs Heavy Regex">
    At runtime, envtrap does **not** evaluate expensive regular expressions on streaming buffers. Instead, it performs direct substring searches against the registered secret values. This ensures near-zero latency impact on high-throughput Node.js servers.
  </Accordion>

  <Accordion title="1 MB Memory Protection (ContentClamp)">
    To protect the Node.js event loop from memory exhaustion and denial-of-service, content buffers larger than **1 MB** are clamped. Only the first 1 MB of any single payload is scanned.
  </Accordion>

  <Accordion title="TTL Alert Deduplication (DedupCache)">
    To prevent runaway alert storms in high-concurrency loops, duplicate detections of the same secret on the same channel within a **1.5-second TTL window** are suppressed. Only the initial occurrence triggers an alert and enters the report.
  </Accordion>
</AccordionGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.