What is the problem?
There are two problems we met:
There are a branch of features designed privately and implemented without public discussion. There is no truth but only tribal knowledge among those features. For example, Raft Async IO, Cardinality Estimation Enhancement, SPM Enhancement, and so on.
The lack of sharing design and public discussion hurts open source collaboration - contributors can hardly cooperate widely and smoothly, without necessary knowledge and information. For example, when @tangwz tried to understand and potentially participant Cardinality Estimation Enhancement work, the answer is “we discuss privately and develop in close”.
What is proposed to do?
I propose we stick to the design process documented in our development guide and the repository.
The checkpoint is that the final design document checked in the GitHub repository. We do not require the whole story happens on GitHub - you can always have conversations roughly before create a proposal, start a pre-RFC on internals.tidb.io or even a public Google Docs (in English), etc. But you should bring all previous discussions on GitHub for the final design document, squash to a result and keep a link to the original discussion.
Call for Help: What are examples we can learn from?
The proposal above is clear but still far away from practices. I’d like to ask for your helps to collect best design examples which we can learn from and create our own excellent designs.
To narrow the scope of design, let’s focus on database, operator system, compiler, and network.
Current example list (linked to the reply below):
- CockroachDB: Version Migration
- FLIP-76: Unaligned Checkpoints
- TiDB: Placement Rule in SQL
- TiDB: Restructure Tests
- SkyWalking: The first design of Satellite 0.1.0
- PEP-1: PEP Purpose and Guidelines
It’s a natural requirement on running an open-source community that we can track any feature, enhancement, and bugfix lifecycle.
We’re now creating a tracking issue for each task, which is the root of implementation. But still don’t have a strong requirement for “substantial” changes RFCs, which is the root of design.
Any contributor should be able to learn a feature by its design discussions and implementation steps.