How to read Parquet files from S3, Azure and Google Cloud
Parquet Reader Pro opens Parquet files straight from cloud storage. Click Open Cloud, paste a path and pick a connection. The data goes from your provider directly to your Mac, and you get the same table, SQL and export as with a local file. Cloud reading is Pro at $4.99/mo or $49.99/yr, in the Mac app only. Without the app, DuckDB does this from a terminal.
Open it in Parquet Reader Pro
- Click Open Cloud or press β₯βO. Without Pro, this opens the subscription screen.
- Paste a path. There is no bucket browser, so you need to know it.
- Pick a connection, or click Add a connection⦠to create one.
- Click Open. The table works like a local one: SQL, statistics, export.
- Click Save as table to keep it. A saved cloud table can be joined with a local file in SQL.
What a path looks like
s3://bucket/2026/orders.parquet one file
s3://bucket/2026/ every Parquet file under this folder
s3://bucket/2026/*.parquet every file that matches
gs://bucket/2026/orders.parquet Google Cloud Storage
az://account/container/orders.parquet Azure
On Azure you can also paste an https:// link from the portal or an abfss:// path. A folder reads as one table, the same way a local partitioned folder does.
Try it on public data
Microsoft publishes New York taxi trips in a public container, so this needs no account with them:
az://azureopendatastorage/nyctlc/yellow/puYear=2019/puMonth=1/*.parquet
Paste the path, click Add a connectionβ¦, choose Public container, click Save connection, then Open. You get 7,664,528 rows and 23 columns.
Ways to connect
- AWS: your AWS profile, including one set up with
aws sso login, access keys, or a public bucket. - S3-compatible: access keys and a server URL, for MinIO, Cloudflare R2, Backblaze, Wasabi or Ceph.
- Google Cloud: your
gcloudsign-in, a service account key, HMAC keys, or a public bucket. - Azure: Sign in with Microsoft, an account key, a SAS token, a service principal, or a public container.
Secrets stay in your Mac's Keychain and go only to the provider they belong to. Your AWS and gcloud folders are read, never changed, and you pick each one once. Access is read-only, and your data never passes through our servers.
What can go wrong
- An expired AWS SSO sign-in. The app does not refresh it. Run
aws sso loginagain. - Azure says no access. Your account needs a data role such as Storage Blob Data Reader. Owner alone is not enough. Your company may also have to approve the app once.
az loginis not used. Choose Sign in with Microsoft instead.credential_processdoes not work. The app cannot run other programs. Paste temporary keys instead.
Speed depends on your connection and on how far you are from the data. Your provider may also charge for requests and data transfer, as it does for any other tool.
Other ways to read from the cloud
DuckDB command line, S3
This reads a public AWS dataset, so it needs no keys:
duckdb -c "SET s3_region='us-west-2'; SELECT count(*) AS tiles FROM read_parquet('s3://ookla-open-data/parquet/performance/type=fixed/year=2024/quarter=1/*.parquet')"
βββββββββββ
β tiles β
β int64 β
βββββββββββ€
β 6655986 β
βββββββββββ
DuckDB downloads its httpfs extension for this. For a private bucket, add your credentials with CREATE SECRET.
DuckDB command line, Azure
Azure needs the azure extension and a secret, even for public data:
duckdb -c "INSTALL azure; LOAD azure; CREATE SECRET pub (TYPE azure, CONNECTION_STRING 'AccountName=azureopendatastorage'); SELECT vendorID, count(*) AS trips FROM read_parquet('az://nyctlc/yellow/puYear=2019/puMonth=1/*.parquet') GROUP BY vendorID ORDER BY trips DESC"
ββββββββββββ¬ββββββββββ
β vendorID β trips β
β varchar β int64 β
ββββββββββββΌββββββββββ€
β 2 β 4649800 β
β 1 β 2937920 β
β 4 β 76808 β
ββββββββββββ΄ββββββββββ
Note that the path differs from the one in the app. Here the account is in the secret, so the path starts at the container.
pyarrow
import pyarrow.parquet as pq
from pyarrow import fs
s3 = fs.S3FileSystem(anonymous=True, region="us-west-2")
path = "ookla-open-data/parquet/performance/type=fixed/year=2024/quarter=1/2024-01-01_performance_fixed_tiles.parquet"
f = pq.ParquetFile(path, filesystem=s3)
print(f.metadata.num_rows, f.metadata.num_columns) # 6655986 11
table = pq.read_table(path, filesystem=s3, columns=["avg_d_kbps"])
Reading the row count takes only the footer, and columns keeps the read to what you ask for. Leave anonymous=True out to use your own AWS credentials.
What is free and what is Pro
The cloud is Pro as a whole, a single cloud file included: $4.99/mo or $49.99/yr, as a subscription through the Mac App Store version, and you can cancel anytime. It is not in the Windows and Linux apps.
Local files stay free: full SQL, joins between files, schema, column statistics and export.
Get Parquet Reader
Cloud reading is Pro: $4.99/mo or $49.99/yr. Local files stay free. macOS 13 or later.
Get it on the App Store