SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

Alvito - An Algorithm Visualization Tool for Python

Utils for streaming large files (S3, HDFS, gzip, bz2...)

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

A library on top of either pex or conda-pack to make your Python code easily available on a cluster

Immutable and grow-only Pandas-like DataFrames with a more explicit and consistent interface.

Kafka Connect FileSystem Connector

Data-frames & arrays on Clojure

Web tool for Kafka Connect |

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

Data and tooling to compare the API surfaces of various array libraries.

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

The Universal Storage Engine

Read and write Neuroglancer datasets programmatically.

A Multi-Master Distributed Caching Layer for Amazon S3.

locopy: Loading/Unloading to Redshift and Snowflake using Python.

A command line security audit tool for Amazon Web Services

Painless iOS over-the-air enterprise distribution

🔦 Motion detecting security camera using a raspberry pi, webcam, and slack

Amazon S3 bucket spelunking!