Big Log Explorer: Code Examples

Kubernetes Pod Logs

A crashing pod scrolls kubectl logs past too fast. That is the moment Kubernetes logs become hard, because they stream to your terminal, they rotate away, and a single command shows one pod when the problem spans a whole deployment. The usual fix is to dump the logs to a file with kubectl logs and read them somewhere better. Big Log Explorer is that somewhere. Export a deployment's logs with a label selector, drop the file in, and if your app logs JSON, which is the Kubernetes recommendation, the tool parses the timestamp, level, and message from each line automatically.¹ Consequently, a stream that vanished off your terminal becomes a searchable, filterable, chartable file. You can isolate errors across every replica, find when a crash loop began on the time chart, and cluster the repeating messages that a live tail never lets you count.

Exporting pod logs to a file

The first step is getting the logs out of the cluster and into a file. kubectl logs writes a pod's log to standard output, so redirecting it to a file is as simple as adding a redirection. For a single pod that has already crashed, the previous container's logs matter most, which the -p flag retrieves.² Adding --timestamps prefixes each line with an RFC 3339 time, and --since bounds the export to a recent window so the file stays manageable.

Capturing a whole deployment

A real incident rarely stays on one pod, so you usually want every replica at once. A label selector with -l gathers logs from all pods matching a label, and --prefix tags each line with its pod name so you can tell replicas apart after merging.² Combining these gives one file spanning the deployment, such as kubectl logs -l app=api --prefix --timestamps --since=1h. Consequently, dropping that file into Big Log Explorer lets you search and filter across all replicas together, which is exactly the view kubectl logs cannot give you live. Furthermore, because the export is a plain file, you can capture it once and analyze it repeatedly without holding a connection to the cluster open.

Why JSON logging pays off here

Kubernetes guidance is to log structured JSON to stdout,¹ and that choice determines how much the tool can do for you. When each line is a JSON object, Big Log Explorer reads the timestamp, level, and message from named fields, so the export arrives with the time chart populated and the level pills working. A plain-text log still opens and searches, but it may lack a parseable timestamp or level, which limits charting and filtering. Consequently, the payoff for JSON logging shows up precisely when you need it most, during triage of a large export.

Furthermore, because the container runtime wraps each stdout line into its own JSON record on the node,³ some export paths give you doubly structured data, and reading the application log through kubectl logs unwraps the runtime layer³ and leaves your application JSON on each line. The practical takeaway is simple: log JSON from your app, export with timestamps, and the tool has everything it needs to parse, chart, and cluster.

Reading the wrapped runtime layer

When the node captures stdout, it wraps each application line in its own JSON record, which is why some export paths look doubly structured. Going through kubectl logs strips that outer wrapper, so what lands in your file is the application JSON, not the node's envelope. That matters because the tool keys off the application fields, and a leftover runtime layer would push the real timestamp and level one level deeper than the parser expects. Knowing the export path you used tells you which layer you are actually reading, so you can confirm the right fields are on each line.

Debugging a crash loop

A crash loop is the canonical Kubernetes problem, and a merged log is the fastest way to understand one. When a pod restarts repeatedly, each incarnation writes the same startup sequence followed by the same fatal error, so the log fills with near-identical repetitions. Reading them individually tells you nothing about scale or timing.

Finding when the restarts began

Big Log Explorer turns that repetition into signal. The Patterns panel clusters the recurring fatal message into one high-count template, so you see immediately how many times the pod died and with what error. The time chart, meanwhile, plots the restarts as a regular series of spikes, and the interval between them often matches the backoff the kubelet applies to a crash-looping container.⁴ Consequently, you can read both the cause, in the clustered error, and the cadence, in the chart, from the same file. Furthermore, switching the ERROR pill on and searching for a container name isolates one replica's failures when a selector pulled in several, which separates a single bad pod from a systemic problem across the deployment.

The same clustering helps when the error is not a clean crash. A pod that logs a warning then hangs, or that returns errors on a fraction of requests, leaves a less regular signature, and the panel still surfaces the recurring shape among the noise. Triage in Kubernetes is mostly pattern recognition, so the faster you can see the repeated message, the sooner you know whether one pod or the whole deployment is in trouble.

When to use this

Use this when kubectl logs is too fast, too narrow, or too transient to read live. Export a deployment's logs to a file with a label selector and timestamps, then analyze the merged result in the browser. It suits debugging crash loops, tracing an error across replicas, and post-incident review of a captured window. JSON application logs give the richest parsing.

Notes

kubectl logs only returns logs the node still retains. The kubelet rotates container logs, commonly at a default of around 10 MB per file with a handful of rotations kept, so very old lines may already be gone before you export. For anything beyond that retention, a cluster-level log collector such as an EFK or Loki stack holds the history, and you can export from there instead. Big Log Explorer analyzes whatever file you produce, whether it came from kubectl logs or a collector's export.

Examples

kubectl logs api-7d9f -p --timestamps > api-crash.log

kubectl logs -l app=api --prefix --timestamps --since=1h > api-1h.log

kubectl logs api-7d9f --all-containers --timestamps > pod-all.log

Try in the tool

Docker Container Logs

Docker writes container logs through a logging driver. By default that driver is json-file, which captures everything a container sends to stdout and stderr and stores it on the host as newline-delimited JSON, one object per line holding the message, the stream, and an RFC 3339 timestamp.¹ The docker logs command reads those files back for a container. That default is convenient, but the on-disk wrapper is not the shape you want to analyze directly, because your application's real message sits inside a log field rather than at the top level. The cleaner path is to export with docker logs, which prints each line as your application actually wrote it. Consequently, if your app logs JSON or ISO-timestamped lines, dropping that export into Big Log Explorer gives you full timestamp and level parsing. From there the usual filtering, charting, and clustering apply to any container's output.

The json-file driver and where logs live

Understanding Docker logging starts with the default driver. When you run a container without specifying otherwise, Docker uses the json-file driver, which writes each stdout and stderr line to a file on the host under the container's directory in the Docker data root. Every entry in that file is a JSON object with three keys: log, holding the actual line your program emitted, stream, marking stdout or stderr, and time, an RFC 3339 timestamp the daemon added. The docker logs command simply reads and unwraps those files.

Consequently, you have two ways to get the data out, and they are not equivalent. Reading the raw json-file gives you Docker's wrapper around each line, while docker logs gives you the line as your application wrote it. Furthermore, the json-file driver is also where unbounded log growth comes from, since without rotation limits it fills the host disk, which is why production hosts configure max-size and max-file on the driver.¹

Reading the raw file versus docker logs

The two export paths look similar but hand the tool very different shapes. The raw json-file is Docker's object with the message nested under a log key, so a line's real content is one field deep and the level is nowhere to be found. docker logs unwraps that object and prints the message itself, which is the line your application actually wrote. The difference is invisible in the terminal but decisive in the viewer, because only the unwrapped line lets the parser reach your timestamp and level directly.

Exporting the right way for analysis

How you export decides how much Big Log Explorer can parse, so it is worth choosing the command before you capture anything. The goal is to hand the tool your application's own log lines, not the Docker transport wrapper that sits around each one, because only the real message carries the fields the parser needs. A clean export is the difference between a file the tool charts fully and one where your level and timestamp stay hidden, which is why this step shapes everything that follows.

docker logs versus the raw json file

Running docker logs container_name > app.log prints each line exactly as your program emitted it, so a container that logs JSON produces a clean JSONL file the tool parses fully, with the timestamp, level, and message read from your fields. Adding --timestamps prepends Docker's RFC 3339 time to each line, and --since bounds the window.² By contrast, dropping the raw json-file from disk into the tool parses the daemon's time key for the chart but leaves your real message buried inside the log field, with no level. Consequently, docker logs is almost always the better export for analysis. Furthermore, for a multi-container application, docker compose logs gathers every service into one stream, and redirecting that to a file lets you analyze the whole stack together, with each line prefixed by its service name for disambiguation.³

Analyzing a container log once it is open

With a clean export loaded, a Docker log behaves like any other application log in the tool. If the container logs JSON, the level pills separate errors from noise, the time chart shows when a problem started, and the Patterns panel ranks the repeating messages. If the container logs plain text, search and clustering still work over every line even where the timestamp or level is absent.

The journald alternative

Not every host uses json-file. The journald driver sends container output to the systemd journal instead, and you retrieve it with journalctl rather than docker logs.⁴ Exporting from there in JSON, using journalctl with its json output mode, produces one JSON object per entry that Big Log Explorer parses cleanly, including the container name and the message. Consequently, whichever driver a host uses, there is an export path that yields analyzable structured lines. Furthermore, because the tool reads whatever file you give it locally, you can compare a json-file export from one host with a journald export from another in successive sessions without either leaving your machine.

The journald path also shows why the export step matters as much as the analysis. A journalctl JSON export carries the container name as a field, so once it is in the viewer you can filter by container without any prefix hack, whereas the raw json-file would have hidden that detail behind Docker's wrapper. Picking the right command is therefore the same lesson repeated across drivers: give the tool your fields, not the platform's envelope, and the charts and pills light up.

When to use this

Reach for this when a container's logs are too long to read with docker logs alone, or when you need to filter and chart them rather than tail them. Export with docker logs container > file, adding --timestamps and --since for a bounded window, then analyze in the browser. A container that logs JSON gives the fullest parsing, with working level pills and time chart.

Notes

The default json-file driver grows without bound unless you set max-size and max-file, so a long-running container can accumulate gigabytes on the host. When you export such a log, the file may be large, which is exactly what Big Log Explorer is built to handle. Prefer docker logs over reading the raw json-file so your application's real fields, rather than Docker's transport wrapper, reach the tool. If the host uses the journald driver, export with journalctl in JSON mode instead.

Examples

docker logs api --timestamps > api.log

docker logs api --since 1h --timestamps > api-1h.log

docker compose logs --timestamps > stack.log

Try in the tool

Node.js JSON Logs (Pino, Winston, Bunyan)

Your Node service logs JSON by the gigabyte. Pino, Winston, and Bunyan all emit newline-delimited JSON in production,¹ which is perfect for machines and unreadable for a human scrolling a terminal. When you export a busy service's log to a file, you want it parsed, not pretty-printed line by line. Big Log Explorer reads Node JSON logs directly, pulling the timestamp from the time field and the message from the msg field, which Pino, Bunyan, and Winston all use. The one field that needs attention is the level, because Pino and Bunyan write it as a number by default while the level pills expect a label. Configure your logger to emit the string level and the pills light up. Consequently, a production JSONL log from any of the major Node loggers becomes fully searchable, filterable by severity, and chartable over time.

How the tool reads Pino, Winston, and Bunyan

The three dominant Node loggers agree on more than they differ, which is why one tool reads all of them. Pino writes one JSON object per line with a time field in epoch milliseconds and the message in a msg field.¹ Bunyan is similar, with an ISO time field and a msg field.² Winston is configurable, but its common JSON setup uses a message field and, when you add the timestamp format, a timestamp field.³ Big Log Explorer looks for exactly these keys, reading time, ts, timestamp, or datetime for the clock and msg, message, or body for the text.

The numeric level gotcha

Levels are where the loggers diverge in a way that matters here. Pino and Bunyan encode severity as a number by default, so info is 30 and error is 50,² while the level pills match on words like error and warn. A numeric level therefore reads as unclassified, landing lines in the other bucket. The fix is small, because you configure Pino with a level formatter that emits the string label, or set Winston to write a textual level, which it does by default.³ Consequently, with string levels the pills separate errors from noise across the whole file, while the timestamp and message parse correctly regardless.

The same numeric gotcha appears in other frameworks that copy Pino, so the habit of checking your level type pays off beyond Node. If you ever see a large file where every line sits in the other bucket, a numeric level is the most likely cause, and switching to string labels is usually a one-line config change. Treating the level as configuration rather than a log convention is what keeps the pills honest on every export.

Getting the log out of your service

Exporting a Node log well is mostly about not transforming it on the way out. In production, Pino and its peers write raw JSON to stdout, and the standard practice is to let the process manager or container capture that stream. To analyze it, you want that raw JSON in a file, unmodified.

Redirect stdout, avoid the pretty printer

The key is to skip development-time formatters when you export. Tools like pino-pretty reformat JSON into colored, human-readable lines, which is great in a terminal but strips the structure the tool relies on.⁴ For analysis, capture the raw JSON instead, whether by redirecting the process stdout to a file, reading the container log with docker logs, or pulling it from your process manager's log file. Consequently, the file you drop into Big Log Explorer is clean JSONL, and every field your logger wrote is available for search and clustering. Furthermore, because Node services often attach rich context to each line, such as a request ID, a route, and a latency, those custom fields ride along in the raw line and become searchable, letting you trace one request across a large log by its ID.

Tracing requests and errors in a Node log

Once a clean Node JSON log is open, the structure your logger added pays off directly. A per-request log line usually carries a correlation or request ID, so typing that ID into the search box pulls every line for one request across the whole service, even if they are scattered through millions of others. That is the trace a live tail can never assemble. With string levels configured, the ERROR pill then isolates the failures, and the time chart shows when they spiked.

Furthermore, the Patterns panel clusters the repeating messages, which in a Node service often reveals a single throwing route or a dependency timing out far more than its share. Because Node logs frequently embed nested objects for the request and response, the full JSON stays visible in each viewer row, so you read the detail without a separate parser. Between correlation-ID search, level pills, and clustering, a large Node JSON log resolves into answers quickly rather than a scroll.

When nested objects help and hurt

Node loggers routinely attach a full request or response object to a line, and the tool keeps that nesting intact so you can read it in the viewer row. That depth is useful when a failure hides in one field of an embedded object, because you see it without leaving the line. It can also blur the top-level shape the Patterns panel groups on, so a very deep object occasionally splits across templates it should share. The practical habit is to keep the most useful identifiers, a request ID and a status, at the top level of each line so both search and clustering find them reliably.

When to use this

Use this whenever a Node service produces more JSON logs than you can read live. Export the raw stdout JSON to a file, keeping string levels so the pills work, then analyze it in the browser. It suits tracing a request by correlation ID, isolating errors across a busy service, and clustering the messages a Pino or Winston log repeats most. Skip pino-pretty for the export.

Notes

Pino and Bunyan emit numeric levels by default, which Big Log Explorer reads as unclassified rather than as error or warn. To make the level pills work, configure Pino with a level formatter that returns the string label, or rely on Winston's default textual level. The timestamp and message parse regardless, since time and msg are read directly. Avoid exporting through pino-pretty, because its human-readable reformatting removes the JSON structure the tool parses.

Examples

pino({ formatters: { level: (label) => ({ level: label }) } })

node server.js > app.jsonl

winston.format.combine(winston.format.timestamp(), winston.format.json())

Try in the tool

Python Logging Analysis

Python's default log format fights your reading tools. The standard logging module writes lines like 2026-07-09 14:32:05,123 - myapp - WARNING - message, where the level sits in the middle of the line rather than right after the timestamp.¹ A parser that keys on a leading timestamp reads the time but treats the severity as ordinary text, so the level pills stay empty. It is not broken, just positional in a way that hides the level. The clean fix is structured JSON, which python-json-logger and structlog both produce. When your Python service logs one JSON object per line with a level and an ISO timestamp, Big Log Explorer parses all three fields directly. Consequently, an exported Python log gains working level pills, an accurate time chart, and message clustering, instead of a wall of dash-separated text you can only grep.

How default logging parses, and where it falls short

The standard library logging module is where most Python logs begin, and its default text output parses partially in the tool. A typical line leads with an asctime timestamp, which Big Log Explorer reads for the time chart even though Python separates the milliseconds with a comma rather than a dot. What it does not read is the level, because the WARNING or ERROR keyword sits several fields into the line, after the logger name¹, rather than in a bracket right after the timestamp where the parser looks.

Consequently, a default logging file charts on time and searches fully, but every line defaults to the info level regardless of its real severity. Furthermore, the dash-delimited layout is positional, so a message containing its own dashes reads no differently from the field separators. None of this stops you searching or clustering the file, yet it does mean the level pills, one of the fastest triage tools, sit idle until you change the format.

What a fully parsed Python log unlocks

The upside of fixing the format is not just prettier charts, it is the triage speed the level pills give you. Once the severity reads correctly, one click isolates every error across a file that may run to millions of lines, and the Patterns panel ranks the failures by count. A default text log denies you that, leaving the ERROR pill empty and forcing you to grep for words the logger happened to use. Switching to JSON, or to a bracketed level, is therefore the difference between filtering by meaning and scrolling by hand.

Switching to JSON for full parsing

The reliable way to make a Python log fully parseable is to emit JSON with named fields. A JSON line removes all positional ambiguity, since the level is a value under a level key rather than a word in a fixed column. Two libraries make this straightforward without rewriting your logging calls.

python-json-logger and structlog

python-json-logger plugs a JSON formatter into the standard logging module, so your existing logger.info and logger.error calls keep working while the output becomes JSON. Renaming fields so that levelname becomes level and asctime becomes timestamp gives Big Log Explorer exactly the keys it reads.² structlog takes a similar approach with its own processor pipeline, ending in a JSON renderer that emits a level and a timestamp per event.³ Consequently, either library produces a file where the tool reads the severity, the time, and the message directly, and the level pills work. Furthermore, both let you attach structured context, such as a request ID or a user, as additional keys, which then travel with each line and become searchable in the viewer for tracing one request across a large log.

Reading tracebacks and errors

Python errors bring a specific challenge that shapes how you read the log: the traceback. An exception logged with exc_info spans many lines, one per stack frame, and only the first carries the timestamp and level. The frames that follow are indented continuation lines with no timestamp of their own.

Multi-line tracebacks

Big Log Explorer parses each line independently, so the continuation frames of a traceback load as raw text with no timestamp, while the header line carries the level and time. In the viewer they still appear in order, so you read the whole traceback in sequence, but the frames themselves are not individually charted. JSON logging helps here too, because python-json-logger² and structlog³ can serialize an exception into a single field on one JSON line, keeping the whole error as one record. Consequently, a JSON-formatted Python log keeps each error to a single parseable line rather than scattering it across many. Furthermore, with the ERROR pill active the Patterns panel clusters recurring exceptions by their message, so a repeated ValueError or a database timeout surfaces as one high-count template even across a noisy service.

That layout still keeps the traceback readable, because the viewer groups the frames under their header line. You do not lose the connection between the error and its stack frames: the level pill on the header still classifies the whole sequence, and every indented line is searchable exactly as Python wrote it. A multi-line traceback is therefore navigable without changing to JSON, as long as the exported file preserves the line order.

When to use this

Use this when a Python service produces more logs than you can scan, especially during an error investigation. For the fullest parsing, log JSON with python-json-logger or structlog so the level, timestamp, and message all read cleanly. Default text logs still chart on time and search well, but their level pills stay idle. Export the raw output to a file, then filter and cluster in the browser.

Notes

Python's default logging format places the level mid-line, after the logger name, so Big Log Explorer reads the timestamp but defaults the level to info. To activate the level pills, either emit JSON with a level key using python-json-logger or structlog, or format standard logging as a clean ISO timestamp followed immediately by a bracketed level, which the tool detects directly. Python's default comma before the milliseconds is tolerated when reading the time, though a dot or an ISO datefmt is cleaner. Multi-line tracebacks parse per line, so JSON that keeps an exception on one line reads more cleanly.

Examples

JsonFormatter(rename_fields={'levelname': 'level', 'asctime': 'timestamp'})

structlog.configure(processors=[TimeStamper(fmt='iso'), JSONRenderer()])

logging.basicConfig(datefmt='%Y-%m-%dT%H:%M:%S', format='%(asctime)s [%(levelname)s] %(message)s')

Try in the tool

systemd journald Logs

The systemd journal is not a text file. Where syslog and application logs are lines you can cat or tail, the journal is a structured binary store that only journalctl reads directly.¹ That design buys rich indexed metadata on every entry, but it also means you cannot simply drop the journal into a browser tool. You export it first. How you export decides how well it parses, because journalctl can emit plain text, ISO-timestamped lines, or JSON,² and only some of those shapes match what Big Log Explorer reads. Choose an ISO output and the time chart works; reshape the JSON into standard keys and the level pills work too. Consequently, analyzing journald in the browser is less about the tool and more about picking the right journalctl output format, which this guide walks through field by field.

Exporting the journal with journalctl

Everything starts with journalctl, since it is the only supported reader of the binary journal. By default it prints a syslog-like text format with a Mmm dd hh:mm:ss timestamp, the same terse BSD shape that Big Log Explorer does not parse for time. You can narrow what you export with the same flags you use to read live: -u for a unit, --since and --until for a window, and -p for a priority. Redirecting the result to a file gives you something to analyze.

short-iso for a time-charted export

The output format is the lever that matters. Passing -o short-iso rewrites each line to lead with an ISO 8601 timestamp, which the tool reads directly into the time chart, and -o short-iso-precise adds microsecond precision.² This one flag turns a journal export from an unparseable timestamp into a charted one. Consequently, for most journald analysis, journalctl -u myservice -o short-iso --since today is the export to reach for first. Furthermore, because you can scope the export to a single unit and a time window, the file stays small enough to index quickly even on a busy host.

Why raw journalctl JSON needs reshaping

The obvious move, journalctl -o json, produces JSON but not the JSON the tool expects. The journal names its fields in uppercase with leading underscores: MESSAGE holds the text, PRIORITY holds a numeric syslog severity, and __REALTIME_TIMESTAMP holds the time in epoch microseconds.³ Big Log Explorer looks for lowercase timestamp, level, and message keys, so it matches none of these, which leaves each line parsed as JSON but unclassified, with the whole object treated as the message.

Raw journal field names and the parser gap

Consequently, raw journal JSON gives you searchable, clustered lines but no time chart and no level pills. The fix is to reshape the objects into the keys the tool reads, which a small jq filter does in the pipeline. Mapping __REALTIME_TIMESTAMP to a timestamp, translating PRIORITY to a level word, and copying MESSAGE to message produces exactly the structure the tool parses fully. Furthermore, that same filter is where you can drop the noisy journal fields you do not need.

Analyzing a journal export

Once you have a well-shaped export, a journal log behaves like any other structured log in the tool. An ISO or reshaped-JSON export charts on time, so you see when a service started failing, and with a level present the ERROR pill isolates the failures. The Patterns panel clusters the repeating messages a systemd unit emits, which often surfaces a restart loop or a recurring dependency error immediately.

Per-unit and per-boot filtering

Much of the useful scoping happens at export time with journalctl itself. Restricting to a unit with -u keeps the file focused on one service, and -b limits it to the current boot, which is invaluable when diagnosing a problem that started after a reboot. You can also pre-filter by priority with -p err to export only the more severe entries.³ Consequently, you arrive in Big Log Explorer with a file already narrowed to the relevant service and window, then use search, level pills, and the time chart to finish the investigation. Furthermore, because the export is a plain file, you can capture a journal snapshot during an incident and re-analyze it later without shell access to the host.

Capturing a snapshot before a reboot or a rolling restart also preserves the exact timeline that caused the failure. If you export after the event, the missing lines are gone. A pre-reboot journal export therefore becomes a static record of every failure, warning, and launch message that led up to the moment you restart.

When to use this

Use this when the logs you need live in the systemd journal rather than in flat files. Export with journalctl, choosing -o short-iso for a time-charted file or reshaping -o json with jq for full level pills, and scope it with -u, -b, or --since. It suits diagnosing a failing systemd unit, reviewing a boot, or capturing a journal snapshot to analyze offline.

Notes

The journal is binary, so you must export through journalctl before analysis. journalctl -o json emits objects with uppercase, underscore-prefixed fields such as MESSAGE, PRIORITY, and __REALTIME_TIMESTAMP, which do not match the lowercase timestamp, level, and message keys Big Log Explorer reads, so raw journal JSON loads unclassified. Prefer -o short-iso for a time-charted export, or reshape the JSON with a jq filter that maps __REALTIME_TIMESTAMP to a timestamp, PRIORITY to a level word, and MESSAGE to message. Scope exports with -u, -b, and --since to keep the file small.

Examples

journalctl -u myservice -o short-iso --since today > svc.log

journalctl -o json | jq -c '{timestamp: (.__REALTIME_TIMESTAMP|tonumber/1000000), message: .MESSAGE}' > svc.jsonl

journalctl -b -p err -o short-iso > boot-errors.log

Try in the tool

AWS CloudWatch Logs

You need a CloudWatch log group in front of you. Reading logs through the AWS console works for a few lines, but scrolling a busy log group, correlating across streams, or reviewing an incident window is slow and clumsy there. Exporting the events to a file and analyzing them locally is faster and cheaper than running query after query. CloudWatch makes this convenient in one respect, because each log event is wrapped with a lowercase timestamp in epoch milliseconds and a message field,¹ which happen to be exactly the keys Big Log Explorer reads. Export the events as one JSON object per line and the tool charts them on time immediately. Consequently, a CloudWatch log group becomes a searchable, filterable, chartable file in the browser. If your application logs JSON inside each message, unwrapping that envelope unlocks the level pills too.

Exporting events from a log group

There are a few ways to get CloudWatch events into a file, and the CLI is the most scriptable. The AWS CLI exposes log events through commands that return the raw events, each carrying a numeric timestamp and a message, so you extract them into a file you can analyze. Scoping the export to a time range and a log group keeps it small and relevant.

filter-log-events and the tail command

Two commands cover most needs. aws logs filter-log-events returns matching events as JSON with lowercase timestamp and message fields,¹ and piping the events array through jq into one object per line gives Big Log Explorer a clean JSONL file where the timestamp charts directly. aws logs tail streams a group and, with --format short, prints ISO-timestamped text lines that also chart on time,² which is handy for a quick capture. Consequently, filter-log-events suits a precise, scripted export while tail suits a fast one. Furthermore, both let you bound the window with start and end times or a --since duration,² so you export only the incident period rather than the whole retention of the group.

Unwrapping the message for level pills

CloudWatch gives you the timestamp for free, but the level takes one more step. The envelope CloudWatch adds has a timestamp and a message, and that message field is whatever your application wrote, which may itself be a JSON object with its own level and fields. Reading the envelope alone therefore charts on time but leaves the severity locked inside the message string.

Two layers of JSON

When your app logs structured JSON, each CloudWatch event is JSON wrapping JSON. To get the level pills working, unwrap the inner payload so the application's own fields sit at the top level of each line. A jq filter that takes each event and emits its message parsed as JSON³ produces exactly that, giving you the application timestamp, level, and message the tool reads directly. Consequently, the difference between a time-charted export and a fully classified one is a single unwrapping step in the pipeline. Furthermore, if your application logs plain text rather than JSON, the envelope timestamp still charts the events, and search plus clustering work over every message, so you lose only the level pills and nothing else.

The wrapper event also keeps the logStreamName and timestamp readable at the top level, which means you do not lose the stream identity when you unwrap the message. Exported events still travel together as one JSONL file, so the viewer can merge many streams into one timeline while the search box filters by the original log stream name.

Analyzing across streams and the whole group

One of the reasons to pull CloudWatch into a local tool is to see across streams at once. A log group holds many streams, one per container, instance, or Lambda invocation, and the console tends to show them one at a time. Because filter-log-events spans the whole group by default, a single export can merge every stream into one file, which is exactly the cross-stream view an incident needs.

Merging streams into one view

Once it is open, the logStreamName that CloudWatch includes on each event stays in the line, so searching a stream name isolates one source when you need to, while leaving the merged view available. Furthermore, the time chart over a whole-group export shows when a problem appeared across all streams together, and dragging to that window narrows every stream at once. Consequently, a CloudWatch export in Big Log Explorer gives you the aggregate picture and the per-stream detail from the same file, without paying for a Logs Insights query on each iteration of your investigation.

When to use this

Use this when reading a CloudWatch log group through the console is too slow or too costly to iterate on. Export the events with aws logs filter-log-events or aws logs tail, bounding the window, then analyze the file locally. It suits incident review across many streams, tracing a request through a log group, and clustering the messages a service repeats. Unwrap JSON messages for level pills.

Notes

CloudWatch wraps each event with a lowercase timestamp in epoch milliseconds and a message field, which match the keys Big Log Explorer reads, so an exported JSONL of events charts on time without extra work. The level, however, lives inside the message when your application logs JSON, so unwrap the inner payload with jq to activate the level pills. aws logs tail --format short is a fast ISO-timestamped alternative. Bound every export with a start and end time or --since, since a busy log group can hold far more than you want to index at once.

Examples

aws logs filter-log-events --log-group-name /app/api --start-time 1720526400000 | jq -c '.events[]' > api.jsonl

aws logs filter-log-events --log-group-name /app/api | jq -c '.events[].message | fromjson' > api.jsonl

aws logs tail /app/api --since 1h --format short > api-1h.log

Try in the tool