Query Avro Files with SQL in Your Browser

Query Avro Files with SQL in Your Browser

When a streaming system needs schema and bytes to travel together, Avro keeps them in one self-describing file.¹² The SQL Workbench decodes the Avro schema and row data in the browser, then hands the result to DuckDB so you can query Kafka exports and pipeline dumps with full SQL.

Why Avro is common in streaming data pipelines

The Apache project designed Avro for schema evolution in streaming systems where producers and consumers may operate at different schema versions.² The schema is stored as a JSON block inside the Avro container header, which makes the file self-describing: a reader can inspect column names and types without a separate schema registry. That property explains Avro's dominance in Apache Kafka ecosystems. AWS Glue Schema Registry supports Avro as one of its streaming schema data formats.³ Avro uses a row-oriented binary layout, which is efficient for write-heavy streaming workloads where records arrive one at a time. Because the schema is embedded, a .avro file you export from a Kafka topic or a Confluent export contains everything the workbench needs to load and query it correctly.

How the workbench loads Avro files

Avro files go through a JavaScript bridge that reads the container header, extracts the embedded schema, and decodes the binary row data. The schema sets the column names and types for the DuckDB table: Avro string fields become VARCHAR, int fields become INTEGER, long fields become BIGINT, and union types that allow null become nullable columns. After decoding, the workbench hands the rows to DuckDB, so you query Avro data with the same SQL as Parquet or CSV. The table name comes from the file stem.

Row-oriented loading trade-off

Because Avro is row-oriented, the JavaScript bridge reads the entire file during loading; column pruning is not available the way it is with Parquet. Practically, this means query performance scales with file size more directly than columnar formats, but moderate-sized Avro exports (up to a few hundred megabytes) load in a few seconds. Once loaded into DuckDB, however, query execution itself is fast because the data is in a structured tabular format that the engine can scan efficiently.

For workloads that only need a few columns from a very large Avro export, the full-file read means you pay the load cost regardless of how little you query, which is the main practical limit relative to columnar formats. The trade-off is usually acceptable because the files arrive from streaming exports that are already bounded in size, and the structured DuckDB table you get at the end supports fast filtering and aggregation on every column.

Querying Kafka exports and pipeline dumps

Kafka topic exports saved as Avro typically contain a key field, a value struct, and metadata like partition and offset.⁴ The value struct becomes a nested DuckDB STRUCT column. Access its fields with dot notation: SELECT value.user_id, value.event_name FROM topic_export.⁵ For schema evolution scenarios where newer records have fields older records lack, those missing fields become NULL in the query result. Filtering by timestamp works once you identify the timestamp field name: WHERE strptime(created_at, '%Y-%m-%dT%H:%M:%SZ') > '2026-01-01'. Export the query result to CSV, JSON, Excel, or Parquet to move it into another tool. You can also filter by partition or offset to isolate a specific slice of the topic, which is useful when debugging a particular producer batch without reprocessing the entire file.

Inspecting Avro schema before writing queries

Inspecting the Avro schema is the first step after loading, because column types can differ from what you might expect from the original Kafka message schema. Run DESCRIBE on the loaded table immediately to see what types DuckDB assigned. The JavaScript bridge maps Avro string to VARCHAR, long to BIGINT, double to DOUBLE, boolean to BOOLEAN, and union types like ["null","string"] to nullable VARCHAR.

Avro logical types and how they appear in DuckDB

Avro logical types are annotations on top of primitive types. A timestamp-millis logical type on a long field stores a Unix epoch timestamp in milliseconds. Depending on how the JavaScript Avro decoder handles this logical type, the field may arrive as BIGINT (the raw millisecond value) or as a formatted string. Check DESCRIBE output and, if the field arrives as BIGINT, convert it: to_timestamp(created_at_ms / 1000.0) produces a readable DuckDB TIMESTAMP.

Reading nested Avro records

Nested Avro records become DuckDB STRUCT columns, so field access follows the same dot-notation pattern you use for Parquet nested types. If a Kafka value contains user.id and user.email, DESCRIBE shows the nested structure before you write SELECT value.user.id, value.user.email FROM topic_export. For deeply nested records with multiple levels, you can chain dot notation to reach any depth, and DESCRIBE output helps you verify the exact field names and types before constructing those paths.

Exporting Avro data to Parquet for analytical use

Exporting Avro query results to Parquet converts a row-oriented binary format into a columnar one that DuckDB queries more efficiently on future sessions. After the Avro file loads through the JavaScript bridge, the data is a standard DuckDB view, and any query result is exportable to Parquet via the Export button.⁶

For Avro files from Kafka topic exports, the common pattern is to flatten nested struct fields into top-level columns before exporting: SELECT value.user_id, value.event_type, value.timestamp FROM kafka_export. Then click Export and choose Parquet. The exported Parquet file has a simpler, flat schema that DuckDB queries faster in future sessions because it avoids the struct field access step, which is why flattening Avro structs before export pays off on later queries.

When to use this

Use this when you have a .avro file from a Kafka export, a Confluent topic dump, or an AWS Glue job and want to inspect its schema and run SQL against the event records. Drop the file into the workbench and run DESCRIBE right after loading, since that is the fastest way to confirm what types the JavaScript bridge actually assigned before you write a query against them.

Examples

DESCRIBE events;

SELECT event_type, COUNT(*) AS n
FROM events
GROUP BY event_type
ORDER BY n DESC;

SELECT value.user_id,
       value.action,
       value.session_id
FROM kafka_export
LIMIT 50;

SELECT *
FROM events
WHERE created_at BETWEEN '2026-03-01' AND '2026-03-31';