Big Log Explorer

How it works

Most log viewers load the entire file into memory before showing anything. That caps their practical file size at whatever RAM the browser will hand over. Big Log Explorer takes a different path. A Web Worker reads the file in 1 MB chunks via the browser's File API.¹ Workers run scripts in background threads without interfering with the user interface.² The parsed records are written to IndexedDB one batch at a time.³ The browser tab keeps responding while indexing runs in the background.

Once indexing completes, the Worker is terminated and the main thread opens a read-only connection to the same IndexedDB database. The log viewer uses virtual scrolling: only the rows currently on screen plus a small overscan buffer are in the DOM at any time. A spacer div sized to the total line count gives the scrollbar correct proportions across millions of lines.

Getting a stack's logs into the drop zone

Most real 500 MB log files come from a platform, not a laptop: a container orchestrator, a container runtime, a service manager's journal, or a cloud log service. Every one of these offers a way to write its output to a local file, and a local file is all the drop zone asks for. Once the file lands, the format follows the emitter. Structured JSON loggers drop one object per line into the JSON Lines branch, plain application text with a leading timestamp into the ISO 8601 branch, and ingress or web-server access logs into the Common Log Format branch. Nothing about the stack changes that routing.

Capture one service or one container at a time rather than a whole cluster's interleaved firehose. Pattern clustering reads best when a file comes from a single emitter, because interleaved producers weave their templates together and blur the counts. The platform's own log-retrieval path is the capture mechanism: Kubernetes documents a built-in way to fetch a pod's logs,⁴ and Docker documents the same for any container it runs,⁵ so getting logs onto disk needs nothing beyond the platform you already operate. Save the retrieved output as a file, drop it here, and the pipeline takes over from that point.

Supported log formats

Format detection happens per line rather than per file, so a single log can freely mix timestamps, levels, and payloads from different emitters without confusing the parser. This lets you drop in a real production export that interleaves application logs, access logs, and the occasional raw exception stack trace. Three structured formats are recognised automatically; any line that matches none of them is kept as raw text so nothing disappears from the viewer.

• JSON Lines (JSONL).⁶ Any line starting with { is parsed as JSON. Timestamp fields (timestamp, time, ts, @timestamp, datetime) and severity fields (level, severity, lvl) are extracted automatically. This covers Bunyan, Winston, Pino, Logstash, and most structured logging formats.
• ISO 8601 log lines. Lines starting with a date-time string in ISO 8601 format are recognised.⁷ They may be optionally followed by a bracketed level like [ERROR]. This covers the default output of most application loggers including Log4j, syslog with ISO timestamps, and cloud provider log exports.
• Common Log Format.⁸ The format used by Nginx access logs and many web-server access logs.⁹ HTTP status codes map to severity: 5xx is error, 4xx is warn, everything else is info.

How each format is recognised

The reader tries the formats in a fixed order for every line, checking the cheapest rule first to keep overhead negligible even across tens of millions of lines. JSON Lines wins if a line opens with a curly brace and parses as valid JSON; ISO 8601 wins if the leading characters encode a date and time; Common Log Format wins if the line matches the request-and-status shape used by web servers.

A fourth rule recognises lines whose leading characters encode a day-named-month-year timestamp such as 15-Mar-2026 or 15/Mar/2026, parsing the timestamp and treating the line as info-level. Each branch pulls out the same three fields, a timestamp, a severity level, and a message, so the viewer can sort, filter, and chart lines regardless of which producer emitted them; lines that win none of the rules are kept as raw text and still appear in the viewer and the pattern panel.

Choosing between overlapping formats is handled by specificity rather than by file extension, so a log that begins as plain Common Log Format but suddenly switches to JSONL mid-stream is indexed correctly without any manual configuration. The formats list is intentionally small: it covers the majority of operational logs while keeping the per-line detection fast enough to sustain line rates of hundreds of thousands per second inside a Worker thread.

Formats the rules cover beyond the big three

Two formats the named list never mentions still parse cleanly. The combined log format extends Common Log Format with the referrer and user-agent fields after the status code,⁸ and because the CLF rule matches through the status field, those trailing fields never break the match: a combined line charts and level-maps exactly like a plain one. Syslog lines in the modern RFC 5424 shape begin with an ISO timestamp, and the ISO rule recognises them for exactly that reason, since it keys on the leading date-time rather than on the format's name. The big three were always a simplification of what the rules accept.

The honest boundary is classic syslog. Lines that lead with a month-first timestamp and no year, the RFC 3164 shape, match none of the rules and land as raw text: fully visible in the viewer under the ALL pill, counted in the pattern panel, and absent from the time chart because no timestamp parsed. Lines whose severity parses as other likewise appear under ALL only, because the pill row carries no OTHER entry. Neither behavior loses data; both only narrow where the chart and the pills can take you. A raw-text file still opens, indexes, and clusters normally.

Pattern clustering

The Patterns panel groups log lines by template so you can see, at a glance, which messages dominate a noisy file. Each message is normalised by replacing variable parts, including URLs, IP addresses, UUIDs, hex strings, file paths, floats, and integers, with placeholders such as <url> and <n>; the curly braces in JSON keys are replaced by a single placeholder too, so two otherwise identical lines with different IDs collapse into one cluster. Lines that produce the same normalised string are counted together.

How the patterns filter the viewer

The top 50 patterns by count are shown in the panel, each row carrying the template and the number of lines it matched. The Filter button on a row narrows the log viewer to lines matching that template, and that filter combines with any active text search or level filter already in place, so you can isolate one recurring error across a five-hundred-megabyte file without downloading anything. The cluster counts themselves are computed once, over the whole file, during indexing; they stay whole-file counts while the viewer narrows, and the Worker that built them is terminated as soon as indexing completes.

Privacy

Log files regularly contain hostnames, user IDs, request paths, and authentication tokens, which is why the entire pipeline of file reading, line parsing, IndexedDB indexing, and filtering is designed to run inside your browser tab. No data reaches any server. A dedicated Web Worker reads and parses the file on a background thread without interfering with the user interface, so the heavy lifting never blocks the tab you are reading.²

Where the data is stored and when it disappears

The parsed records are written to an IndexedDB object store that lives only for the lifetime of the page, scoped to the current tab's origin. The IndexedDB database is deleted when you close or reload the tab, so nothing written during one session is carried into the next. The resulting database is local to your machine and is never uploaded, synchronised, or shared across origins, because IndexedDB follows a same-origin policy and the tool performs no network requests with file contents at any point.³

Time-chart navigation for incident triage

During an incident, narrowing down when a problem started is often the first step toward finding what caused it. The time-series chart at the top of the log viewer plots line counts per time bucket across the full span of the log file. A spike in error-level lines at a specific time stands out visually even before you read a single log entry. Dragging across the chart to select a time window immediately filters the log viewer to lines from that period, letting you read a few hundred lines in context rather than scrolling through millions. Clicking outside the highlighted region restores the full log.

Combining the time filter with a level filter and a text search lets you isolate a specific error class within a short window with a few clicks. For example: select the five-minute window around a spike, switch the level pill to ERROR, and search for a service name. The viewer then shows only error-level lines mentioning that service in those five minutes. This layered filtering needs no query language. The pattern list stays a whole-file view while you work, so the template that dominates the selected window is found by reading the narrowed viewer and refining the search, not by watching pattern counts move. Because the entire log is stored locally in IndexedDB, all filtering happens on your machine regardless of file size, which keeps iterative narrowing practical for logs several hundred megabytes long.

What search can and cannot do

The search box matches a plain, case-insensitive substring against each raw line after a half-second debounce, and the match combines with the level pill, the active pattern, and the selected time window. Each added filter narrows the same line set rather than starting the search over. Searching for an error string with the ERROR pill pressed is the fastest route from a noisy file to one failure class, because the substring names the failure and the pill removes everything the failure is not. Nothing you type needs escaping or quoting, which keeps the tool honest about being a viewer first.

Every character you type is literal. There are no regular expressions, no wildcards, and no boolean operators, so a search for error.*timeout looks for that exact run of characters rather than a pattern. The narrowed result lives in the viewer and nowhere else: there is no export or download of the filtered lines, no way to save the selection to disk from this tool. When you need the narrowed set somewhere else, the reproducible artifact is the source file plus your filter choices, and re-creating the view is a few clicks. The viewer is the destination by design.