← All guides Pro

How to read Parquet files from S3, Azure and Google Cloud

Parquet Reader Pro opens Parquet files straight from cloud storage. Click Open Cloud, paste a path and pick a connection. The data goes from your provider directly to your Mac, and you get the same table, SQL and export as with a local file. Cloud reading is Pro at $4.99/mo or $49.99/yr, in the Mac app only. Without the app, DuckDB does this from a terminal.

Open it in Parquet Reader Pro

  1. Click Open Cloud or press βŒ₯⌘O. Without Pro, this opens the subscription screen.
  2. Paste a path. There is no bucket browser, so you need to know it.
  3. Pick a connection, or click Add a connection… to create one.
  4. Click Open. The table works like a local one: SQL, statistics, export.
  5. Click Save as table to keep it. A saved cloud table can be joined with a local file in SQL.

What a path looks like

s3://bucket/2026/orders.parquet      one file
s3://bucket/2026/                    every Parquet file under this folder
s3://bucket/2026/*.parquet           every file that matches
gs://bucket/2026/orders.parquet      Google Cloud Storage
az://account/container/orders.parquet    Azure

On Azure you can also paste an https:// link from the portal or an abfss:// path. A folder reads as one table, the same way a local partitioned folder does.

Try it on public data

Microsoft publishes New York taxi trips in a public container, so this needs no account with them:

az://azureopendatastorage/nyctlc/yellow/puYear=2019/puMonth=1/*.parquet

Paste the path, click Add a connection…, choose Public container, click Save connection, then Open. You get 7,664,528 rows and 23 columns.

Ways to connect

Secrets stay in your Mac's Keychain and go only to the provider they belong to. Your AWS and gcloud folders are read, never changed, and you pick each one once. Access is read-only, and your data never passes through our servers.

What can go wrong

Speed depends on your connection and on how far you are from the data. Your provider may also charge for requests and data transfer, as it does for any other tool.

Other ways to read from the cloud

DuckDB command line, S3

This reads a public AWS dataset, so it needs no keys:

duckdb -c "SET s3_region='us-west-2'; SELECT count(*) AS tiles FROM read_parquet('s3://ookla-open-data/parquet/performance/type=fixed/year=2024/quarter=1/*.parquet')"
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  tiles  β”‚
β”‚  int64  β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ 6655986 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

DuckDB downloads its httpfs extension for this. For a private bucket, add your credentials with CREATE SECRET.

DuckDB command line, Azure

Azure needs the azure extension and a secret, even for public data:

duckdb -c "INSTALL azure; LOAD azure; CREATE SECRET pub (TYPE azure, CONNECTION_STRING 'AccountName=azureopendatastorage'); SELECT vendorID, count(*) AS trips FROM read_parquet('az://nyctlc/yellow/puYear=2019/puMonth=1/*.parquet') GROUP BY vendorID ORDER BY trips DESC"
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ vendorID β”‚  trips  β”‚
β”‚ varchar  β”‚  int64  β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ 2        β”‚ 4649800 β”‚
β”‚ 1        β”‚ 2937920 β”‚
β”‚ 4        β”‚   76808 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Note that the path differs from the one in the app. Here the account is in the secret, so the path starts at the container.

pyarrow

import pyarrow.parquet as pq
from pyarrow import fs

s3 = fs.S3FileSystem(anonymous=True, region="us-west-2")
path = "ookla-open-data/parquet/performance/type=fixed/year=2024/quarter=1/2024-01-01_performance_fixed_tiles.parquet"

f = pq.ParquetFile(path, filesystem=s3)
print(f.metadata.num_rows, f.metadata.num_columns)    # 6655986 11

table = pq.read_table(path, filesystem=s3, columns=["avg_d_kbps"])

Reading the row count takes only the footer, and columns keeps the read to what you ask for. Leave anonymous=True out to use your own AWS credentials.

What is free and what is Pro

The cloud is Pro as a whole, a single cloud file included: $4.99/mo or $49.99/yr, as a subscription through the Mac App Store version, and you can cancel anytime. It is not in the Windows and Linux apps.

Local files stay free: full SQL, joins between files, schema, column statistics and export.

Get Parquet Reader

Cloud reading is Pro: $4.99/mo or $49.99/yr. Local files stay free. macOS 13 or later.

Get it on the App Store