Gizzard (Scala framework)

Gizzard was an open source sharding framework to create custom fault-tolerant, distributed databases. It was initially used by Twitter and emerged from a wide variety of data storage problems. Gizzard operated as a middleware networking service that ran on the Java Virtual Machine. It managed partitioning data across arbitrary backend datastores, which allowed it to be accessed efficiently.[2][3] The partitioning rules were stored in a forwarding table that maps key ranges to partitions. Each partition managed its own replication through a declarative replication tree. Gizzard handled both physical and logical shards. Physical shards point to a physical database backend whereas logical shards are trees of other shards.[4] In addition Gizzard also supported migrations and gracefully handled failures. The system was made eventually consistent by requiring that all write operations are idempotent and commutative. As operations fail they are retried at a later time. Gizzard is available at GitHub and licensed under the Apache License 2.0.