In this recent post, we have used Polars and DuckDB to convert a large CSV file to Parquet in steaming mode – and Python.
Different people have contacted me and asked: “and in R?”
Simple answer: We have DuckDB, and we have different Polars bindings. Here, we are using {polars} which is currently being overhauled into {neopandas}.
Click through for the comparison.