Proposal: Unify TiDB/TiKV real-time communication channels

#1

Background

The TiDB('s) Internals forum is a great place for async communication. It runs well, but the real-time communication for TiDB/TiKV is still in chaos.

For a long time in history, TiDB/TiKV uses slack. We have tidb-community and tikv-wg. In the meantime, PingCAP uses WeChat Enterprise. So the community doesn’t know what happens inside PingCAP. And the slack is not so active.

For now, PingCAP switches to Lark. It’s better than WeChat, but still not so well for the community. We can’t establish a group/chat between the community and core contributors.

The good news is Lark intends to build an open platform, so it provides public APIs as described in docs. And our great open source community has built SDKs like https://github.com/galaxy-book/feishu-sdk-golang

So it’s possible to connect lark to the matrix, slack, telegram, and so on.

Proposal

So I propose to unify TiDB/TiKV real-time communication channels into the matrix and connect to lark and slack.

Matrix is an open network for secure, decentralized communication.

  • It has a secure, decentralized open standard
  • It is licensed under Apache 2.0
  • Both server and client is open source
  • It’s in active development
  • …

It required gitter and provides a bridge to most IM platforms: like IRC, slack, telegram, discord, skype, even WeChat (need extra setup).

Matrix is a great place to be the communication center. All our developers can use whatever tools they want to join in.

Retional

What’s the benefit for developers in PingCAP?

We need real-time communication.

Neither GitHub issue nor internal.tidb.io is not good for detailed problem discussion. With this unified communication channel, we can set up a more fast and open communication workflow.

Think about this bad pattern:

image

In our new communication design, we will:

  • Start a new room at scale
  • Schedule a quick 10 minutes meeting
  • Link the room’s permanent link back

Like:

image
image.png731×180 14.5 KB

We can keep a traceable record while solving problems. It’s important for the community to grow up.

How about private chats in PingCAP?

The async communication channel is set up for tidb / tikv developers. We don’t need to connect all internal rooms to the matrix.

For example, sig-copr, wg-xxx is great to be connected to the matrix. But the rooms with PingCAP business is not fit.

Why not Telegram, Discord, or Xxxx?

There are bridges for telegram and discord, and easy to connect. The most important part of this proposal is to decide where the center is. And which, how many tools to connect is not the core problem, we can leave them to the further.

How about WeChat?

It depends.

Here are two solutions:

Why not Slack?

Oh, It’s too expensive.

image

image
image.png1234×141 10.8 KB

The price mode of Slack is not suitable for open-source communities. And the free version can’t preserve all chatting history.

Thanks for any comments!

3 Likes
#2

So what’s the difference from slack? I mean you still need people to use another “new” communication tool?

#3

First of all, I’m not enforcing everyone to use another new communication tool.

All the work is lied down to our community teams, they need to connect different rooms together. As a normal users, they can use any tool they like. Slack? Telegram? Discord? Matrix? All of them are OK to use, we will discuss just like we sit in the same room. They don’t need to care about which tool others are using.

So this proposal intends to empower our users. They don’t need to register a new slack account anymore if they already have a other platform’s account.

#4

It’s necessary for real-time communication.
Slack or lark is no-active. People just open them when they need help or working.
People can’t see the information in time.
With the WeChat group, everyone can see the relevant information when they open WeChat, and the timeliness will be greatly improved

#5

I have updated the proposal just now, adding a new section about Why not Slack?, worth a reading.

1 Like
#6

I think this is expected. When not working, people may be unwilling to see such information and should not be bothered…

I think Slack/lark do belong to real-time/sync communication. This forum/GitHub issue are async.

#7

If someone only uses Slack for tidb, probably he will seldom open slack. The case becomes worse when pingcap uses lark. The community doesn’t know what happens in pingcap, and pingcap developers may seldom check slack. That’s why slack is not active.

So I think bridging is a good idea. (people can choose a relatively frequently used tool

2 Likes
#8

Yes, what’s worse is the avalanche effect.
Less active people in slack, fewer people opening the slack, and thus the slack will be more inactive.

#9

I think the name is a bit ambiguous where “internal” is almost “kernel” instead of “private”, so “Internal TiDB” is incorrect, “TiDB('s) Internals” is correct.

1 Like
#10

Updated as described.

#11

I have thought about instant messaging system but ended up with it is unnecessary. We cannot have two truth sources and everything just happened on GitHub (at least for now). I create this forum to host low traffic (comparing with issues / PRs) open-ended discussion, where discussion grouped in topic are significant.

I’m open to the community for any kind of communication so if you have an idea, just go ahead.

Slack works well expect it cannot handles all history. It is fine for now as we use it as IM instead of truth source. If you like matrix, (I like Gitter, too), just create a channel / workspace and trying to collect people and so-called “real-time communication” there. No objection from the community for it obviously.

If you guys gather people and create valuable discussions, create a pull request to the README of pingcap/tidb or TiDB Dev Guide or anywhere you’d like to advertise your channel. Just do it.

1 Like
#12

Gitter is Matrix. The core contributor of Matrix (Element) required Gitter and merged them together.

The core idea of this proposal is bridging, we need to bridge people together especially contributors from PingCAP.

In order to archive that, we will need:

  • A bridge between Lark and Matrix
    • Which requires the admin (maybe) permission of Lark org
    • A new appservice at matrix side (need development work)
  • A bridge between Slack and Matrix
    • It’s much easier hence matrix has builtin slack support
    • We need the admin permission of the existing slack channels to connect.

Yes, we can start by creating a new channel at matrix without bridging but it is indeed unnecessary because it’s just another TiDB Contributor Club chat group.

One more thing, Lark has an article about Slack Connector. This feature is implemented by zapier which is not free to use.

#13

I don’t think we should do it this way. If we have a forum which cannot bridge people, a “new” solution, cannot also.

The truth I observed is that,

  1. Contributors from PingCAP (I don’t like this concept, though) don’t expose design and progress among the community. So you don’t know where TiDB to go. Bad case as https://github.com/pingcap/tidb/issues/26085
  2. The voice from contributors drag those who from PingCAP go to community is still weak. As you can see, @skyzh asked for an explanation of changing CI pipeline, and thus we get the reason more publicly and regard this kind of actions cannot be done silently - our contributors put an eye on it.

So, instead of bridging things or doing pivotal tools change, create value and force contributors from different background go to community and share values.

Of course you can build a chat room and trying to bridge channels. But I don’t think it is in an effective direction.

Yes that’s it.

#14

Hi, with the help from @tison and @gingerkidney, we have connected #tidb-chinese together successfully!

Since now, we can discuss problems together in Chinese via tools we prefer:

We have discussed Proposal: Move parser back to pingcap/tidb recently, welcome join in!

1 Like