Nothing Special   »   [go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Consider removing badgerdb #260

Open
faddat opened this issue Jun 4, 2022 · 3 comments
Open

Consider removing badgerdb #260

faddat opened this issue Jun 4, 2022 · 3 comments

Comments

@faddat
Copy link
Contributor
faddat commented Jun 4, 2022

It seems unlikely that badger will continue to be maintained in the future.

I think this leaves us (tendermint, tm-db, cosmos-sdk) with few reasonable choices for databases moving forward:

  • badgerdb (but we maintain it)
  • pebble
  • bolt
  • goleveldb

These made the cut because all are high performance key value stores written in native go. If I had to choose between them, my choice would be pebbledb, but @marbar3778 notes that geth uses goleveldb, but quite differently from how we do it. Most notably, goleveldb in geth splits the store into a long-term (generally read only) section and short term (more changes happen here) section. This is an oversimplification of their approach but I think that this is notable moving forward.
This was referenced Jun 4, 2022
Closed
Closed
@dwedul-figure
Copy link

In some testing I did a couple months ago using Cosmos-SDK v0.45, our blockchain was unusable with boltdb as the database backend for all of the Cosmos-SDK and Tendermint DBs. After converting my node data to boltdb, it would take 10+ seconds to replay a block that only took 6 seconds to create. So it would never have caught up. I'm guessing most of that time was due to bolt using a single file for the entire database, and the application db/file was 100+ Gb. I can't remember how big the other dbs were, and maybe bolt works okay for smaller dbs, but it doesn't scale.

During that testing a couple months ago, I found goleveldb to have the best performance (even better than cleveldb). But again, this testing was done where all dbs were using the same backend.

The drawback to badgerdb is the 2.5x storage space needed. At that extra cost, I would have thought it would have better performance. My testing was mostly focused on writes, and badgerdb was in the middle for that.In theory, though, it should outperform the others when it comes to reading/querying, but I haven't done any testing for that.

@faddat
Copy link
Contributor Author
faddat commented Jul 21, 2022

Furthermore badger DB is abandonware.

@joesixpack
Copy link
joesixpack commented Sep 2, 2022 •
edited
Loading

And boltdb isn't?

Benchmarks: https://github.com/alphayan/kvbench/tree/ae37778240eba3d48a2c3ffeae1c33d00a63e875

I think taking the CockroachDB approach of customizing your own backend makes the most sense as Pebble is hardly blockchain specific.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@faddat @joesixpack @dwedul-figure