Proposal: Remove always failed CI

#1

How about removing always failed CI like

  • idc-jenkins-ci-tidb/code-coverage: failed for bind: address already in use
  • idc-jenkins-ci-tidb/common-test: failed for errors like role1@% is not granted to root
  • idc-jenkins-ci-tidb/integration-common-test: failed for dial tcp 127.0.0.1:4001: connect: connection refused
  • idc-jenkins-ci-tidb/integration-copr-test: failed for Unsupported cast from Enum to Enum

Those tests are keeping failed for a long time:

image
image.png1204×1749 356 KB

It seems no one really care about them, ever for committers:

image
image.png1971×1131 261 KB

It’s OK to merge a PR even when it’s unit test failed for


[2021-12-31T05:48:03.924Z] FAIL: handle_test.go:987: testSerialStatsSuite.TestAnalyzeGlobalStatsWithOpts2
[2021-12-31T05:48:03.924Z] 
[2021-12-31T05:48:03.924Z] handle_test.go:1003:
[2021-12-31T05:48:03.924Z]     s.checkForGlobalStatsWithOpts(c, tk, "test_gstats_opt2", "test_gstats_opt2", "p1", 1, 50)
[2021-12-31T05:48:03.924Z] handle_test.go:933:
[2021-12-31T05:48:03.924Z]     c.Assert(numBuckets, GreaterEqual, buckets-delta)
[2021-12-31T05:48:03.924Z] ... compare_one int = 0
[2021-12-31T05:48:03.924Z] ... compare_two int = 15

Why don’t we just remove them and add them back when they are (actually) ready?

#2

We should fix it, instead of remove it!

1 Like
#3

For unit test, there is a ongoing improvement plan for it, see https://github.com/pingcap/tidb/issues/30822

If you’re interested, you can make ut and fix some test cases, to make this progress faster @Xuanwo

1 Like
#4

I agree with your comments with 100 percents.

But the tests for master branch keeps failed at least since Oct 13, 2021 (sorry the commits history is so long that I can’t found the real start). Is there any work going on for it?

#5

For errors like bind: address already in use, they are more like a infra related issues? Any plan to fix them?

#6

Some of the test cases are pooly written and they use some random port and conflict…
The key to solve it is git blame to catch the original author, then create a bug issue and assign to him/her.

According to our current developing rule, if someone has too many bug issue assigned, he/she would have to stop the developing and fix bugs first.

3 Likes