Allow the ratchet job to pass if tests pass which are not listed as expected to pass #3369

New Issue

exarkun · 2020-08-06T13:56:24Z

exarkun commented

2020-08-06 13:56:24 +00:00

While porting fileutil (/tahoe-lafs/trac-2024-07-25/issues/8008) the ratchet job complained that some non-fileutil tests started passing. These tests were incidental to the porting effort and they may not actually represent successfully/completely ported implementation and tests. It was a little surprising to have to add them to the ratchet-passing list and a little confusing to review those additions. The tests may also flap back to failing in other porting effort bumps them in the wrong way. This would create friction for that future porting effort.

The downside of allowing this is that the CI job won't force porters to add the tests they've ported to the list so it will require more care during that part of the process. Of course, if tests are missed then they can always be added to the list at a later stage when someone realizes they've actually been ported. This doesn't seem like that bad of a downside.

While porting fileutil ([/tahoe-lafs/trac-2024-07-25/issues/8008](/tahoe-lafs/trac-2024-07-25/issues/8008)) the ratchet job complained that some non-fileutil tests started passing. These tests were incidental to the porting effort and they may not _actually_ represent successfully/completely ported implementation and tests. It was a little surprising to have to add them to the ratchet-passing list and a little confusing to review those additions. The tests may also flap back to failing in other porting effort bumps them in the wrong way. This would create friction for that future porting effort. The downside of allowing this is that the CI job won't force porters to add the tests they've ported to the list so it will require more care during that part of the process. Of course, if tests are missed then they can always be added to the list at a later stage when someone realizes they've actually been ported. This doesn't seem like *that* bad of a downside.