New storage engine for TiDB?

Just found this repo, looks interesting… https://github.com/pingcap/tidb-engine-ext

1 Like

What’s the difference between this project and brson’s engine traits? Looks like they both could add support for multiple engines in TiKV.

I think the name should be tiflash-proxy, it is a fork of TiKV that adds support TiFlash needs, mainly learner related logics.

As @tison said, it’s actually a wrapper of TiKV’s Raft component, with which you can implement your own storage engine that consumes TiKV Raft log and snapshot.

Currently it only allows you to join a TiKV cluster as a Learner, so you have to implement Follower Read to make your storage engine readable.

1 Like

And AFAIK, engine traits is a feature allows you to replace RocksDB with your customized storage engine. It’s more like the concepts of plugable storage engine in MySQL.

1 Like

Actually I’m curious isn’t it a feature should / already provided by TiKV? IIRC current TiKV allows you to join a cluster as a learner.

Correct.

Whatever, you have to do some wrapping work to get up with TiKV Raft component, while tidb-engine-ext has done this for you. It provides a c dynamic library and exposes some useful APIs.

Why expose c api instead of rust library? The latter is not only easier to achieve but also popular in new databases.

Oh, I got it, it’s not a new thing.

I got you got it.:slightly_smiling_face: