try switching from coveralls.io to codecov.io #2389

New Issue

tahoe-lafs · 2015-03-10T17:07:36Z

daira commented

2015-03-10 17:07:36 +00:00

The codecov.io reports apparently include branch coverage, which I consider essential, but coveralls does not support. The changes needed seem completely straightforward.

The only disadvantage would be that the historical coverage data wouldn't be copied across, but I think that's fine.

The codecov.io reports apparently include branch coverage, which I consider essential, but [coveralls does not support](https://github.com/lemurheavy/coveralls-public/issues/31). The [changes needed](https://github.com/jantman/pypuppetdb-daily-report/pull/39/files) seem completely straightforward. The only disadvantage would be that the historical coverage data wouldn't be copied across, but I think that's fine.

tahoe-lafs added the

labels 2015-03-10 17:07:36 +00:00

tahoe-lafs added this to the soon (release n/a) milestone 2015-03-10 17:07:36 +00:00

zooko commented

2015-03-10 17:19:33 +00:00

When doing this, please see if you can figure out how to elicit information about which lines had changes in their coverage between versions. Coveralls just reports the change in the number of lines that are covered, which is not really useful information.

When doing this, please see if you can figure out how to elicit information about *which lines had changes in their coverage* between versions. Coveralls just reports the *change in the number of lines that are covered*, which is not really useful information.

daira commented

2015-03-12 18:28:49 +00:00

"This application will be able to read and write all public and private github repo data."

Scratch that then.

"This application will be able to read and write all public and private github repo data." Scratch that then.

tahoe-lafs added the

wontfix

label 2015-03-12 18:28:49 +00:00

tahoe-lafs closed this issue

2015-03-12 18:28:49 +00:00

exarkun commented

2020-12-29 17:52:47 +00:00

(https://www.tahoe-lafs.org/trac/tahoe-lafs/ticket/3385) was a duplicate of this.

exarkun commented

2020-12-29 17:57:34 +00:00

coveralls.io no longer requests the "repo" scope (I believe that is the scope that corresponds to the description "This application will be able to read and write all public and private github repo data.".

Instead, it requests:

Read org and team membership, read org projects
Access commit status
Access user email addresses (read-only)

This is substantially less invasive so it seems reasonable to consider this switch again.

My current interest stems from codecov behavior of reporting results early, before all jobs have run, which invariably makes the report look extremely bad. I've tried fixing this with "wait_for_ci: true" (https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3568) but to no avail.

coveralls.io no longer requests the "repo" scope (I believe that is the scope that corresponds to the description "This application will be able to read and write all public and private github repo data.". Instead, it requests: * Read org and team membership, read org projects * Access commit status * Access user email addresses (read-only) This is substantially less invasive so it seems reasonable to consider this switch again. My current interest stems from codecov behavior of reporting results early, before all jobs have run, which invariably makes the report look extremely bad. I've tried fixing this with "wait_for_ci: true" (<https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3568>) but to no avail.

exarkun removed the

wontfix

label 2020-12-29 17:57:38 +00:00

exarkun reopened this issue

2020-12-29 17:57:38 +00:00

exarkun commented

2020-12-29 20:15:12 +00:00

Here's a setback. To support collecting coverage data from more than one job, you need to manually deliver a "finished" notification to coveralls.io. With some annoying manual work it's possible to do this correctly for either CircleCI or GitHub Actions but delivering the finished hook only after both CI providers are done is a lot harder.

I guess you'd need a job on one of them that could find the corresponding jobs on the other and wait? And only after that job (and all other local jobs) finish would you deliver the finished notification.

I don't feel like working on this any more today.

Here's a setback. To support collecting coverage data from more than one job, you need to manually deliver a "finished" notification to coveralls.io. With some annoying manual work it's possible to do this correctly for _either_ CircleCI _or_ [GitHub](wiki/GitHub) Actions but delivering the finished hook only after _both_ CI providers are done is a lot harder. I guess you'd need a job on one of them that could find the corresponding jobs on the other and wait? And only after that job (and all other local jobs) finish would you deliver the finished notification. I don't feel like working on this any more today.

exarkun commented

2020-12-29 20:16:09 +00:00

For reference https://docs.coveralls.io/parallel-build-webhook

For reference <https://docs.coveralls.io/parallel-build-webhook>

exarkun commented

2020-12-29 20:19:19 +00:00

Aha. I just realized I read the ticket summary backwards. sigh.