How to open a folder of partitioned Parquet files
Parquet Reader Pro opens a whole folder of Parquet files as one table. Click Open Folder, pick the top folder, and every file below it becomes one dataset, with hive partition folders as real columns. Pro is $4.99/mo or $49.99/yr, and opening a single file stays free. Without the app, DuckDB does the same with a glob.
What a partitioned folder is
Spark, Hive and most pipelines do not write one big Parquet file. They write a tree:
sales/
โโโ year=2025/month=12/part-0.parquet
โโโ year=2026/month=1/part-0.parquet
โโโ year=2026/month=2/part-0.parquet
Three things about this layout catch people out:
key=valueis a convention between engines. It is not part of the Parquet format, so a tool has to choose to read it.- The partition column is usually not in the files. Open
part-0.parquetalone andyearandmonthare not there. They exist only in the path. - Files drift. A column added last month exists in the new files only.
Open it in Parquet Reader Pro
- Click Open Folder or press โโงO. Without Pro, this opens the subscription screen.
- Choose the top folder, here
sales. Subfolders are included, and both.parquetand.pqfiles are read. - The table shows the rows of every file.
yearandmonthare ordinary columns you can filter and sort. - Schemas are merged by column name. When a file does not have a column, its rows show NULL there.
- In the SQL Query panel,
parquet_tablenow means the whole folder.
SELECT year, month, count(*) AS orders, sum(amount) AS revenue
FROM parquet_table
GROUP BY year, month
ORDER BY year, month
โโโโโโโโโฌโโโโโโโโฌโโโโโโโโโฌโโโโโโโโโโ
โ year โ month โ orders โ revenue โ
โ int64 โ int64 โ int64 โ double โ
โโโโโโโโโผโโโโโโโโผโโโโโโโโโผโโโโโโโโโโค
โ 2025 โ 12 โ 1 โ 10.0 โ
โ 2026 โ 1 โ 1 โ 20.0 โ
โ 2026 โ 2 โ 1 โ 30.0 โ
โโโโโโโโโดโโโโโโโโดโโโโโโโโโดโโโโโโโโโโ
Other ways to read the folder
DuckDB command line
duckdb -c "SELECT year, month, count(*) AS orders, sum(amount) AS revenue FROM read_parquet('sales/**/*.parquet', hive_partitioning = true, union_by_name = true) GROUP BY year, month ORDER BY year, month"
** walks the subfolders. A filter on a partition column, such as WHERE year = 2026, lets it skip whole folders.
pandas
import pandas as pd
df = pd.read_parquet("sales/")
print(df.dtypes)
pandas reads the partition folders too. In my run, year and month came back as category, not integers, so cast them before doing arithmetic.
The column that disappears
I tested this with a coupon column that exists only in the newest file. union_by_name = true kept it and filled the older rows with NULL. Without that option, DuckDB returned the rows with no coupon column and no warning. pd.read_parquet("sales/") dropped it the same way. Versions were DuckDB 1.5 and pandas 3.1 with pyarrow 26.
So whichever tool you use, check the column list against what you expect before trusting a result from a folder.
What is free and what is Pro
Folders are the Pro part: $4.99/mo or $49.99/yr, as a subscription through the Mac App Store version, and you can cancel anytime.
A single file stays free, of any size: full SQL, schema, column statistics, and export to CSV, JSON or Markdown. If you only need one file from the tree, open that file. Just remember that its partition columns will not be there.
If the folder is in S3, Azure or Google Cloud, see the guide to reading Parquet from the cloud.
Get Parquet Reader
Folders are Pro: $4.99/mo or $49.99/yr. Single files stay free. macOS 13 or later.
Get it on the App Store