← All guides Pro

How to open a folder of partitioned Parquet files

Parquet Reader Pro opens a whole folder of Parquet files as one table. Click Open Folder, pick the top folder, and every file below it becomes one dataset, with hive partition folders as real columns. Pro is $4.99/mo or $49.99/yr, and opening a single file stays free. Without the app, DuckDB does the same with a glob.

What a partitioned folder is

Spark, Hive and most pipelines do not write one big Parquet file. They write a tree:

sales/
โ”œโ”€โ”€ year=2025/month=12/part-0.parquet
โ”œโ”€โ”€ year=2026/month=1/part-0.parquet
โ””โ”€โ”€ year=2026/month=2/part-0.parquet

Three things about this layout catch people out:

Open it in Parquet Reader Pro

  1. Click Open Folder or press โŒ˜โ‡งO. Without Pro, this opens the subscription screen.
  2. Choose the top folder, here sales. Subfolders are included, and both .parquet and .pq files are read.
  3. The table shows the rows of every file. year and month are ordinary columns you can filter and sort.
  4. Schemas are merged by column name. When a file does not have a column, its rows show NULL there.
  5. In the SQL Query panel, parquet_table now means the whole folder.
SELECT year, month, count(*) AS orders, sum(amount) AS revenue
FROM parquet_table
GROUP BY year, month
ORDER BY year, month
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ year  โ”‚ month โ”‚ orders โ”‚ revenue โ”‚
โ”‚ int64 โ”‚ int64 โ”‚ int64  โ”‚ double  โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚  2025 โ”‚    12 โ”‚      1 โ”‚    10.0 โ”‚
โ”‚  2026 โ”‚     1 โ”‚      1 โ”‚    20.0 โ”‚
โ”‚  2026 โ”‚     2 โ”‚      1 โ”‚    30.0 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Other ways to read the folder

DuckDB command line

duckdb -c "SELECT year, month, count(*) AS orders, sum(amount) AS revenue FROM read_parquet('sales/**/*.parquet', hive_partitioning = true, union_by_name = true) GROUP BY year, month ORDER BY year, month"

** walks the subfolders. A filter on a partition column, such as WHERE year = 2026, lets it skip whole folders.

pandas

import pandas as pd

df = pd.read_parquet("sales/")
print(df.dtypes)

pandas reads the partition folders too. In my run, year and month came back as category, not integers, so cast them before doing arithmetic.

The column that disappears

I tested this with a coupon column that exists only in the newest file. union_by_name = true kept it and filled the older rows with NULL. Without that option, DuckDB returned the rows with no coupon column and no warning. pd.read_parquet("sales/") dropped it the same way. Versions were DuckDB 1.5 and pandas 3.1 with pyarrow 26.

So whichever tool you use, check the column list against what you expect before trusting a result from a folder.

What is free and what is Pro

Folders are the Pro part: $4.99/mo or $49.99/yr, as a subscription through the Mac App Store version, and you can cancel anytime.

A single file stays free, of any size: full SQL, schema, column statistics, and export to CSV, JSON or Markdown. If you only need one file from the tree, open that file. Just remember that its partition columns will not be there.

If the folder is in S3, Azure or Google Cloud, see the guide to reading Parquet from the cloud.

Get Parquet Reader

Folders are Pro: $4.99/mo or $49.99/yr. Single files stay free. macOS 13 or later.

Get it on the App Store