Scrub PII from JSON Logs and API Payloads

Scrub PII from JSON Logs and API Payloads

JSON is the default format for API logs, webhook payloads, and database document exports.¹ A single API response can contain dozens of PII fields such as user IDs linked to emails, nested address objects, credit card metadata, and session tokens, often deeply nested in ways that make manual review impractical.

When you paste JSON payloads into AI tools for debugging, analysis, or transformation assistance, every nested field travels with the paste. The scrubber detects sensitive values regardless of nesting depth, replacing them with tokens while preserving the full JSON structure that makes the payload meaningful for the downstream task.

PII in JSON structures

API responses nest PII at multiple levels. A user object might contain an email at the top level and a phone number nested under the address object, with a session JWT embedded in a separate metadata field. Consequently, a manual inspection of a 50-field JSON document is both time-consuming and error-prone. The scrubber processes the full text, detecting email patterns, JWT headers (eyJ prefix), IPv4 addresses, and credential formats across every field regardless of nesting.² JSON structure, including keys, brackets, and non-PII values, is preserved after tokenization.

You should pay special attention to nested objects that contain both identifiers and authentication data. A single API response might include a user profile with an email address, a payment method object with a credit card number, and a session object with a JWT token. All three are PII, and all three are detected by the scrubber regardless of how deeply they are nested. The scrubber does not need to understand the JSON schema; it scans the raw text for pattern matches, so even unconventional nesting structures or dynamically generated field names do not prevent detection.

Common JSON sources that contain sensitive data

API logs are the most frequent source: request logs include caller IPs and authorization headers; response logs include user data and session tokens. Webhook payloads from payment providers include card metadata. Database document exports from MongoDB or Firestore include user profile documents with email addresses, phone numbers, and address fields. Furthermore, CI/CD pipeline outputs, error tracking payloads, and observability traces (OpenTelemetry, Datadog) frequently include request context with internal IPs and auth tokens.

You should treat any JSON that has passed through a production system as potentially containing PII.³ Error tracking payloads from Sentry, Datadog, or Rollbar frequently include request context with user IPs, session tokens, and authenticated user email addresses. Observability traces from OpenTelemetry or AWS X-Ray include service-to-service call metadata that may contain internal hostnames and API keys. Before pasting any of these JSON sources into an AI tool for analysis, run them through the scrubber to catch the embedded identifiers that are easy to overlook in deeply nested trace data.

Preserving JSON validity after scrubbing

The scrubber performs string replacement, not JSON parsing. It replaces raw string values that match PII patterns. Because it targets values rather than structure, keys, brackets, commas, and colons are untouched. Yet some edge cases, such as a URL string that contains an email address or a description field that includes a phone number in prose, will be tokenized in place. The resulting JSON remains syntactically valid and can be parsed, diffed, or imported into another tool after scrubbing.

After scrubbing, you can validate the JSON by pasting it into any JSON parser or validator. The token format [EMAIL_1] is a valid JSON string value, so the scrubbed output parses identically to the original from a structural perspective.⁴ For workflows where you need to diff the scrubbed JSON against the original (for example, to verify that only PII fields were changed), a text diff will show exactly which values were replaced. Non-PII fields, numeric values, boolean values, and null values are all preserved exactly as they were in the original.

Handling arrays and high-volume JSON payloads

JSON arrays of objects are processed by the scrubber in a single pass across the full text. A 500-element user array containing an email and phone number in each object is scrubbed as fast as a single-object response, because the scrubber performs text pattern matching rather than JSON tree traversal. Nesting depth has no effect on detection accuracy: an email address at five levels of nesting is detected the same way as one at the top level.

For very large JSON payloads (API log exports with thousands of entries), paste the content in segments if your browser slows down during processing. Each segment produces an independent variables file; download and store all segment variables files together so the complete token-to-value mapping is available when you need to restore. Within each segment, the same value always maps to the same token, so [EMAIL_1] in segment one and [EMAIL_1] in segment two represent the same original email address only if the segments were scrubbed in separate passes from separate source texts.

Minified JSON and formatted JSON are both supported

The scrubber processes raw text regardless of formatting, which means you do not need to reformat your JSON before pasting it in. Minified JSON (all on one line with no whitespace) and pretty-printed JSON (indented with newlines) both work correctly because the pattern matching targets values within the string, not structural characters. Neither format produces different detection results, so you can paste API responses directly from your browser's network inspector or from a log file without any preprocessing step.

JWT tokens embedded in JSON API responses

JWT tokens appear frequently in JSON as the value of fields named token, access_token, id_token, authorization, or session. The three-segment eyJ prefix detection the scrubber uses catches all standard JWTs regardless of which JSON field name contains them. Each JWT is replaced with a distinct [JWT_N] token, so multiple JWTs in the same payload are tracked separately in the variables file.

JWTs embed claims in their payload segment. When a JWT contains a user's email address, user ID, or organizational claims, those values are encoded inside the token itself. The scrubber tokenizes the full JWT string, which effectively removes all the claims encoded in it in a single step.² If you need to share the claims structure (to show a JWT's payload to an AI tool for schema review), decode locally, then scrub the JWT claims using a library or jwt.io before pasting, and share only the scrubbed claim structure.

Authorization headers in JSON-formatted HTTP request logs

HTTP request logs formatted as JSON often include an Authorization header field containing a Bearer prefix followed by a JWT or API key. The scrubber detects both the JWT format and the API key patterns in this field regardless of the Bearer prefix. After scrubbing, the log retains the Authorization key and the Bearer prefix with the token replaced by a placeholder, preserving the structural context that shows an authenticated request was made.

NDJSON log output and document database exports

NDJSON (Newline Delimited JSON) is the standard output format for log aggregation tools, MongoDB mongoexport, Elasticsearch exports, and Datadog log archives. Each line in an NDJSON file is a complete JSON object. The scrubber processes NDJSON as plain text: paste the NDJSON content directly and each line's values are detected and tokenized independently. The output is valid NDJSON with tokens replacing sensitive values, preserving the one-document-per-line format that downstream tools expect.

MongoDB mongoexport produces NDJSON by default when exporting a collection.⁵ A user collection export contains one document per line with email, phone, address, and account fields as JSON object values. Pasting a mongoexport output into the scrubber tokenizes all PII across every document in a single pass, giving you a scrubbed export suitable for sharing with a developer or AI analysis tool without transmitting any production user records.

Differencing scrubbed JSON for AI-assisted schema review

Pasting two versions of the same JSON schema (before and after a field addition, for example) into an AI tool for comparison is a common developer workflow. Scrubbing both versions before pasting ensures that example values in the schema do not expose real data. JSON schema comparison requires structural information (key names, value types, array shapes) but never requires the actual example values.⁶ The scrubber removes the values and preserves the structure that makes schema comparison useful.

Because the scrubber preserves keys and structure while replacing only values, the diff shows exactly which fields changed without revealing the sample data behind them. CapyToolkit runs the comparison preparation locally in your browser, so the JSON never leaves your machine during scrubbing and the variables file is the only artifact that maps the tokens back to the real schema examples.

When to use this

Use this before pasting any API payload, webhook body, or JSON document export into an AI tool when the payload contains user data, authentication tokens, or internal service addresses.

Examples

{"id": 42, "email": "[email protected]", "phone": "+14155550100", "token": "eyJhbGciOiJSUzI1NiJ9.eyJ1c2VyX2lkIjoiMTIzIn0.sig"}

{"id": 42, "email": "[EMAIL_1]", "phone": "[PHONE_1]", "token": "[JWT_1]"}

{"event": "charge.succeeded", "customer_email": "[email protected]", "card_last4": "4242", "ip": "203.0.113.5"}

{"event": "charge.succeeded", "customer_email": "[EMAIL_1]", "card_last4": "4242", "ip": "[IP_1]"}