How to query a Parquet file with SQL
Open the file in Parquet Reader, type a query in the SQL Query panel and press βReturn. The file is available as a table called parquet_table. You do not need Python, Spark or a database server, and SQL is free. On the command line, DuckDB does the same in one line.
Run SQL in Parquet Reader
- Open the file. The panel is already filled in with
SELECT * FROM parquet_table LIMIT 100. - Replace the query with your own.
- Press βReturn or click Run Query.
- The result replaces the table. You can search it, copy it or export it. Show Original or ββ§R brings the file back.
Filter and sort
SELECT *
FROM parquet_table
WHERE country = 'DE'
ORDER BY amount DESC
LIMIT 100
Group and aggregate
SELECT country, count(*) AS orders, sum(amount) AS revenue
FROM parquet_table
WHERE status = 'paid'
GROUP BY country
ORDER BY revenue DESC
βββββββββββ¬βββββββββ¬ββββββββββ
β country β orders β revenue β
β varchar β int64 β double β
βββββββββββΌβββββββββΌββββββββββ€
β DE β 1 β 120.0 β
β UA β 1 β 49.9 β
β PL β 1 β NULL β
βββββββββββ΄βββββββββ΄ββββββββββ
Group by month
SELECT date_trunc('month', order_date) AS month, count(*) AS orders
FROM parquet_table
GROUP BY month
ORDER BY month
Join two files
- Open the first file, click Save as table and name it, here
orders. - Open the second file and save it the same way, here
countries. - Use both names in one query.
SELECT c.name, count(*) AS orders, sum(o.amount) AS revenue
FROM orders o
JOIN countries c ON o.country = c.code
GROUP BY c.name
ORDER BY orders DESC, c.name
βββββββββββ¬βββββββββ¬ββββββββββ
β name β orders β revenue β
β varchar β int64 β double β
βββββββββββΌβββββββββΌββββββββββ€
β Germany β 2 β 200.25 β
β Ukraine β 2 β 65.4 β
β Poland β 1 β NULL β
βββββββββββ΄βββββββββ΄ββββββββββ
A saved table points at the file. It is not a copy, so the file has to stay where it is. Saved tables are still in the list after a restart, and Save as table keeps a query result too, so you can query it like any other table.
What the app accepts
- A query must start with
SELECT.WITHis rejected, so write a subquery inFROMinstead. - One statement at a time, read-only. Nothing you type can change the file.
- A query sees your tables only.
parquet_tableis the table on screen, and saved tables go by their names. Functions that read other files, such asread_parquet, are blocked.
SELECT *
FROM (
SELECT country, sum(amount) AS revenue
FROM parquet_table
GROUP BY country
)
WHERE revenue > 50
Other ways to run SQL on Parquet
DuckDB command line
Use the file name where the table name would go:
duckdb -c "SELECT country, count(*) AS orders, sum(amount) AS revenue FROM 'orders.parquet' WHERE status = 'paid' GROUP BY country ORDER BY revenue DESC"
pandas
This one is not SQL, but it answers the same question:
import pandas as pd
df = pd.read_parquet("orders.parquet", columns=["country", "status", "amount"])
paid = df[df["status"] == "paid"]
print(paid.groupby("country")["amount"].agg(["count", "sum"]))
count sum
country
DE 1 120.0
PL 0 0.0
UA 1 49.9
Look at PL. Its only paid order has no amount. SQL counts one order and returns a NULL sum. This pandas version counts non-null values, so it reports zero orders and a sum of 0.0. If a report moved from one to the other, this is where the numbers change.
Reading less of the file
pd.read_parquet loads the columns you ask for into memory, and the filter runs afterwards. DuckDB's read_parquet and polars' scan_parquet push the filter and the column list down into the scan, so they can skip data the query does not touch.
What the free version does not do
SQL is free, without a time limit or a query limit, and so are joins between files, with any number of tables. Reading a whole folder as one table is Pro. See the guide to partitioned folders.
Get Parquet Reader
Free for single files. No account needed. macOS 13 or later.
Get it on the App StoreOn Windows or Linux? Free builds are in the download section.