Big Log Explorer

How it works

Most log viewers load the entire file into memory before showing anything. That caps their practical file size at whatever RAM the browser will hand over. Big Log Explorer takes a different path. A Web Worker reads the file in 1 MB chunks via the browser's File API.¹ Workers run scripts in background threads without interfering with the user interface.² The parsed records are written to IndexedDB one batch at a time.³ The browser tab keeps responding while indexing runs in the background.

Once indexing completes, the Worker is terminated and the main thread opens a read-only connection to the same IndexedDB database. The log viewer uses virtual scrolling: only the rows currently on screen plus a small overscan buffer are in the DOM at any time. A spacer div sized to the total line count gives the scrollbar correct proportions across millions of lines.

Supported log formats

Format detection happens per line rather than per file, so a single log can freely mix timestamps, levels, and payloads from different emitters without confusing the parser. This lets you drop in a real production export that interleaves application logs, access logs, and the occasional raw exception stack trace. Three structured formats are recognised automatically; any line that matches none of them is kept as raw text so nothing disappears from the viewer.

• JSON Lines (JSONL).⁴ Any line starting with { is parsed as JSON. Timestamp fields (timestamp, time, ts, @timestamp, datetime) and severity fields (level, severity, lvl) are extracted automatically. This covers Bunyan, Winston, Pino, Logstash, and most structured logging formats.
• ISO 8601 log lines. Lines starting with a date-time string in ISO 8601 format are recognised.⁵ They may be optionally followed by a bracketed level like [ERROR]. This covers the default output of most application loggers including Log4j, syslog with ISO timestamps, and cloud provider log exports.
• Common Log Format.⁶ The format used by Nginx access logs and many web-server access logs.⁷ HTTP status codes map to severity: 5xx is error, 4xx is warn, everything else is info.

How each format is recognised

The reader tries the formats in a fixed order for every line, checking the cheapest rule first to keep overhead negligible even across tens of millions of lines. JSON Lines wins if a line opens with a curly brace and parses as valid JSON; ISO 8601 wins if the leading characters encode a date and time; Common Log Format wins if the line matches the request-and-status shape used by web servers. Each branch pulls out the same three fields, a timestamp, a severity level, and a message, so the viewer can sort, filter, and chart lines regardless of which producer emitted them.

Choosing between overlapping formats is handled by specificity rather than by file extension, so a log that begins as plain Common Log Format but suddenly switches to JSONL mid-stream is indexed correctly without any manual configuration. The formats list is intentionally small: it covers the majority of operational logs while keeping the per-line detection fast enough to sustain line rates of hundreds of thousands per second inside a Worker thread.

Pattern clustering

The Patterns panel groups log lines by template so you can see, at a glance, which messages dominate a noisy file. Each message is normalised by replacing variable parts, including URLs, IP addresses, UUIDs, hex strings, file paths, floats, and integers, with placeholders such as <url> and <n>; the curly braces in JSON keys are replaced by a single placeholder too, so two otherwise identical lines with different IDs collapse into one cluster. Lines that produce the same normalised string are counted together.

How the patterns filter the viewer

The top 50 patterns by count are shown in the panel, each row carrying the template and the number of lines it matched. Clicking a row filters the log viewer to lines matching that template, and the filter combines with any active text search or level filter already in place, so you can isolate one recurring error across five-hundred-megabyte file without downloading anything. Updating the cluster counts after you apply a time window or a level pill is immediate because the data stays in IndexedDB and the Worker only re-scans the matching records.

Privacy

Log files regularly contain hostnames, user IDs, request paths, and authentication tokens, which is why the entire pipeline of file reading, line parsing, IndexedDB indexing, and filtering is designed to run inside your browser tab. No data reaches any server. A dedicated Web Worker reads and parses the file on a background thread without interfering with the user interface, so the heavy lifting never blocks the tab you are reading.²

Where the data is stored and when it disappears

The parsed records are written to an IndexedDB object store that lives only for the lifetime of the page, scoped to the current tab's origin. The IndexedDB database is deleted when you close or reload the tab, so nothing written during one session is carried into the next. The resulting database is local to your machine and is never uploaded, synchronised, or shared across origins, because IndexedDB follows a same-origin policy and the tool performs no network requests with file contents at any point.³

Time-chart navigation for incident triage

During an incident, narrowing down when a problem started is often the first step toward finding what caused it. The time-series chart at the top of the log viewer plots line counts per time bucket across the full span of the log file. A spike in error-level lines at a specific time stands out visually even before you read a single log entry. Dragging across the chart to select a time window immediately filters the log viewer to lines from that period, letting you read a few hundred lines in context rather than scrolling through millions. Clicking outside the highlighted region restores the full log.

Combining the time filter with a level filter and a text search lets you isolate a specific error class within a short window with a few clicks. For example: select the five-minute window around a spike, switch the level pill to ERROR, and search for a service name. The viewer then shows only error-level lines mentioning that service in those five minutes. This layered filtering does not require any query language, and the pattern panel updates in parallel to show which error templates dominate that window. Because the entire log is stored locally in IndexedDB, all filtering happens instantaneously regardless of file size, making iterative narrowing practical even for logs several hundred megabytes long.