A Solution for TiDB for PostgreSQL

#1

Hello everyone, we are ADDS Lab from Digital China, we strive to bring distributed database solutions to businesses around the globe. This July, we plan to take TiDB for PgSQL to the PingCAP Hacking Camp. Many may be surprised by what we are trying to do – adding PgSQL compatibility to TiDB, a feature that is as in-demand as it is hard to implement.

TiDB is a splendid distributed relational database that we use a lot internally. We have many applications built using PgSQL that we would like to move to TiDB. However, TiDB’s exclusive support for MySQL makes our migration challenging. So naturally, we want a TiDB that talks PgSQL.

The project is still in its early phase. Our first step is to modify TiDB’s protocol layer, making it compatible with PgSQL’s communication protocol. Currently, we have successfully made it possible for a PgSQL client to connect to TiDB and do some basic queries.

TiDB for PgSQL is open-source on GitHub: TiDB-for-PostgreSQL (github.com)

Adding PgSQL compatibility to TiDB will be a mammoth task, so we need some divide and conquer. In this Hacking Camp event, we wish to polish our protocol compatibility further and maybe add few popular PgSQL syntaxes.

You can learn more about our short-term incremental goals for the next three months from this Design Doc.

We also have plans for additional features further down the line:

  1. PgSQL syntaxes
  2. PgSQL table structures
  3. PgSQL system tables and built-in functions
  4. PgSQL datatypes
  5. Other PgSQL features
  6. Related tools in the eco-system

We believe none of them are reachable without support from the open-source community, especially our dear contributors. So we wholeheartedly invite you to join us and work together.

We are students and beginners in developing and managing open-source projects. If you have any advice or suggestions about the project, feel free to open an issue here. If you would like to contribute or wanna talk in private, you can reach us via email at dc.opensource@yungoal.com. We appreciate any suggestions or advice, so don’t hesitate to send them in.

#2

Thanks for sharing your project @pupillord ! Integrate TiDB with PostgreSQL ecosystem sounds interesting to me. I will try to catch up the movement and progress.

However, please keep caution using the “announcement” tag since it means @all . Sharing your excellent project is really encouraged but please leave people interested attracted by the content.

#3

Thank you for your attention and reminder.

#4

TiDB for PostgreSQL July Progress

Previously, we implemented the basic protocol of PostgreSQL in TiDB so that it can successfully connect to the PostgreSQL client and run some SQL. At that time, it was a tentative idea and we wanted to make some estimates, so we did not run any TiDB unit tests when modifying the code and make any modification or addition to the unit tests.

Now, the TiDB for PostgreSQL project is open source on GitHub. In order to ensure the project quality and everyone’s collaborative work efficiency, we need to repair all the previous unit tests and add PostgreSQL compatible unit tests (repairing unit tests takes more work than we thought).

Therefore, the work for the whole of July:

  • Understand TiDB unit test framework and build test environment.
  • Run TiDB unit tests, count the number of failed unit tests, and repair these unit tests that cannot run.
  • Add TiDB for PostgreSQL unit tests.
  • Complete some PostgreSQL syntax parsing in parser and release the alpha version of parser alpha 1.2.

Supplementing and repairing unit tests is a continuous and long-term work, so in upcoming work, we will continue to repair the unit tests of TiDB for PostgreSQL to ensure that all unit tests pass normally.

In addition to repairing unit tests, we also have the following work arrangements for the near future:

  • Improve the PostgreSQL protocol, add more details and functions, and fix some unknown bugs.
  • Improve the execution process of extended queries. There is a big difference between MySQL and PostgreSQL.
  • Continue to modify the parser module and change the basic MySQL syntax to PostgreSQL syntax.
  • Add some support for system tables and system functions.

We refine the above work into issues:
Issues · DigitalChinaOpenSource/TiDB-for-PostgreSQL (github.com)

1 Like