Guide: how to replace circle ci with verify ci

Hi developers!

Some teams have recently encountered limit on the use of circle ci. On behalf of the EE team, we are pleased to announce that jenkins verify ci is now available as a better alternative.

Several projects are now using verify ci as a daily pr quality assurance。The following projects have been tasted so far:

  • [Done] docs
  • [Doing] parser

The following aspects can help you decide whether it is appropriate to use verify ci

  • current cicle ci are often waitting in queue too long to start run
  • current ci only contains some simple shell cmd like make test make lint
  • current ci does not consume excessive resources

Specific steps of migration from circle ci to verify ci

  1. create a pull request on PingCAP-QE/devops-config. Refer to the example to generate the verify.yaml for the project.
  2. request EE team members to review your pr
  3. create a new jenkins pipeline on that using this groovy file pr-verify-check.groovy
  4. test verify ci on your repo
  5. remove your circle ci labelThis text will be hidden from pr merge requied label

verify.config example

verify ci example

update pr commit status after verify ci finished

1 Like

Verify ci on tidb is on the way, but it will not happen in the near future for splitting ut/ft is a heavy work.

Currently tidb use golangci-lint on ci lint check. If you want to add linter to tidb, please refer this config golangci.yaml

More details about golangci-lint linter, please refer

get it, thanks. :+1:

How about using existing enterprise solutions like KodeRover and TeamCity: the Hassle-Free CI and CD Server by JetBrains? No development needed, no manual needed. Developers can just self-service start using them effortlessly.

1 Like

There are some problems I am concerned about:

  • First of all, why Jenkins?
  • Why do we need to design and maintain another new action.yml or travis.yml?
  • Who will maintain the build configs? Maintainers? Committers? Employees from the EE team?
  • Can contributors run those tests on their side?

Can we find better solutions?

For example, run all push / pull requests tests on GitHub actions with self-hosted runners, and run internal/benchmark tests on any internal test infra?

Hi, breeswish, thanks for your suggestions.

  1. EE team member @dianqihanwangzi has been participated in development of Zadig, Zadig is not suitable for our current ci/cd pipeline.
  2. Maybe chosing which ci solution is not the key thing. The key thing is how to make developers self service add or modify them. We will do more research on these enterprise solutions.

You get the point. A DSL (the specific YAML format shown here) is almost a burden to maintain and learn.

You will found it easy at the beginning, but trouble when downstream users ask for more and more “features”.

Even if you take a look at current pipelines to be migrated, it is far more complicated than the bare shellScript case.

Travis CI, GitHub Actions, Circle CI create their own YAML format, all of them are really complicated but still suck on some scenario. I appreciate it you guys develop your own; however, it may not be a good idea to drag our contributors to develop with a demo CI pipeline.

You can hardly resolve the lack of docs by “just writing it”. The youngest solution listed above, GitHub Actions, has a lot of docs that I don’t think you can catch up in even a year. The YAML format won’t be much easier than GitHub Actions as features grown.

Last but not least, integrate with other services and software is the general way to build bigger solution, we don’t have to write anything our own.