Parquet vs CSV: data format differences

CSV stores rows as delimited text. RFC 4180 describes a common form with comma separators, double-quoted fields, and CRLF line endings, but real CSV files vary. The file itself does not declare types, encoding, time zones, null values, or a schema.

Parquet stores data by columns rather than rows. It records column types, groups values into row groups and pages, and can apply compression and encoding per column. Query engines can read only the columns needed for a query, which is why Parquet is common in analytics systems.

CSV is easy to open in a text editor and easy to attach to an email. That readability is a strength for small files and debugging. It is also a weakness because spreadsheet programs may reinterpret IDs, dates, and long numbers on open.

Parquet is not meant to be hand-edited. You inspect it with tools such as DuckDB, Spark, Python, or a Parquet viewer. That makes it less convenient for ad hoc business exchange but better for pipelines where schema and predictable reading matter.

In CSV, 00123, 123, an empty field, and a quoted empty string are all text until a reader guesses their meaning. One tool may infer a number, another may keep text, and Excel may change the display.

Parquet has physical and logical types. A column can be an integer, decimal, timestamp, string, list, or nested struct, depending on the writer. Nulls are represented separately from empty strings. This reduces guessing, but only if the writer chooses the correct schema.

CSV to Parquet requires type decisions. If account numbers, ZIP codes, and identifiers must stay as text, force text before writing. In Python, pandas.read_csv("in.csv", dtype=str) prevents early numeric inference, then you can cast selected columns intentionally.

Parquet to CSV flattens the model. Nested lists and structs need a convention, timestamps need formatting, and binary fields may become unreadable text. Decide how to represent arrays and nulls before exporting to a tool that only understands rows and columns.

Use CSV when the recipient is a person, a spreadsheet, or a simple import form. Use Parquet when the recipient is an analytics database, data lake, or repeatable Python/R/Spark pipeline. For mixed audiences, keep Parquet as the pipeline copy and export CSV slices for review.

For spreadsheet users, an XLSX file may be safer than CSV because it can preserve sheets and text cells more clearly. For databases, SQL dump or Parquet may be safer depending on whether you need schema, indexes, and load behavior.

Schema ownership is the main governance difference. With CSV, every reader can make its own guesses. With Parquet, the writer’s schema travels with the file, so a bad writer decision becomes a durable pipeline decision. Review decimal scale, timestamps, nullable fields, and identifiers before publishing.

Partitioning is also separate from the file format. Data lakes often store many Parquet files under paths such as date=2026-09-10/region=west/. A single Parquet file converted from CSV may be valid, but it is not automatically a well-designed analytics dataset.

When exporting from a database, compare row counts, null counts, and a few typed columns after conversion. Parquet readers can preserve timestamps and decimals, but a careless CSV step before Parquet may already have flattened those values into ambiguous text.

A final format choice should account for who debugs failures. Analysts with SQL tools can inspect Parquet easily through DuckDB or Spark, while business reviewers may need CSV or XLSX extracts to verify disputed rows.