ironarray.io
Compress Better, Compute Larger, Share Faster
👤 Joined on 11 December 2024
Followers
Following
Total Posts
@ironarray.bsky.social@ironarray.bsky.social → @ironarray.io📢 Python-Blosc2 4.13.1 is out! Continuing our "not an island" mood: stream remote datasets (HDF5, Zarr, B2Z) over HTTP/S3 lazily, and compute across formats in one expression: b2z + zarr * h5! Plus tiered caching & Win ARM64 wheels. github.com/Blosc/python... Compress Better, Compute Bigger 🚀
Show Post📢 Python-Blosc2 4.13.0 is out! One lazy API for unite all remote data: browse B2Z/Zarr/HDF5 hierarchies over S3/GCS/HTTP, slice only what you need, cache in RAM or on disk, and export portable snapshots. b2view demo below👇 github.com/Blosc/python... Compress Better, Compute Bigger 🚀
Show Post🚀 New Blosc post: one selective query, 24.3M taxi trips, five tools. ⚡ Cold: CTable/.b2z ~1.9x faster than DuckDB, 9.4x vs pandas 🎯 Block indexes skip ~90% of the data 🤝 Warm: dead heat with DuckDB 💾 Only +2% disk vs Parquet blosc.org/posts/ctable... #Python #DuckDB
Show PostDid you know that in recent Python-Blosc2 4.2.0 we released extremely efficient indexing engines that can store data in (guess what) ...compressed state? As a result, much larger tables can be indexed. Look at how this fares against other good indexing engines in plots below. Enjoy! #TabularData
Show Post