Add code coverage presubmit test by camsim99 · Pull Request #11942 · flutter/packages

camsim99 · 2026-06-18T20:48:04Z

Adds a presubmit test that ensures that the Dart code coverage as evaluated by flutter test --coverage does not fall below a predetermined minimum.

Currently, implements minimums ONLY for camera_android and camera_android_camerax as their current code coverage. The reason I pursued this approach of hardcoding is because a common minimum across packages would be hard to enforce harshly depending on how the package is written. For example, camera_android_camerax wraps native class with Dart, which I believe contributes to its much lower code coverage score. If reviewers are ok to this approach, open to including all packages as their current minimum.

Intended to help agents ensure that while they are making changes, they maintain if not improve the code coverage of the packages they modify.

Pre-Review Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the gemini-code-assist bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling. ↩ ↩²

camsim99 · 2026-06-22T23:58:23Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces a new coverage-check command to the repository tooling, along with CI configuration and a YAML file to define minimum code coverage thresholds for opted-in packages. The command runs tests with coverage, parses the resulting lcov.info file while ignoring generated files, and verifies that the coverage meets the specified minimums. Feedback recommends implementing more defensive YAML parsing to prevent runtime crashes on malformed configuration files, and preserving the coverage directory on failure to assist developers with debugging.

gemini-code-assist · 2026-06-23T00:00:05Z

+    final Directory coverageDir = package.directory.childDirectory('coverage');
+    if (coverageDir.existsSync()) {
+      coverageDir.deleteSync(recursive: true);
+    }
+
+    if (errors.isNotEmpty) {
+      return PackageResult.fail(errors);
+    }
+
+    return PackageResult.success();


Currently, the coverage directory is deleted even when the coverage check fails. This prevents developers from inspecting the generated lcov.info file or generating an HTML report to see which lines/files missed coverage.

Swapping the deletion logic to only run on success improves the developer experience significantly by leaving the coverage artifacts intact for debugging when a check fails.

Suggested change

final Directory coverageDir = package.directory.childDirectory('coverage');

if (coverageDir.existsSync()) {

coverageDir.deleteSync(recursive: true);

}

if (errors.isNotEmpty) {

return PackageResult.fail(errors);

}

return PackageResult.success();

if (errors.isNotEmpty) {

return PackageResult.fail(errors);

}

final Directory coverageDir = package.directory.childDirectory('coverage');

if (coverageDir.existsSync()) {

coverageDir.deleteSync(recursive: true);

}

return PackageResult.success();

I think the test is easily reproduceable locally so not sure we need this.

gemini-code-assist

Code Review

This pull request introduces a code coverage check command (coverage-check) to the tooling, along with CI configuration and tests. The command runs tests with coverage, parses the resulting lcov.info file while ignoring certain generated files, and verifies that the coverage meets specified minimums for opted-in packages. Feedback recommends avoiding the deletion of the coverage directory to assist with local debugging, ignoring .freezed.dart files during coverage calculations, and adding a corresponding test case for this file type.

gemini-code-assist · 2026-06-23T23:15:58Z

+bool _isGeneratedFile(String fileName) {
+  return fileName.endsWith('.g.dart') ||
+      fileName.endsWith('.pb.dart') ||
+      fileName.endsWith('.mocks.dart');
+}


Consider adding .freezed.dart to the list of ignored generated files, as Freezed is a very common code generation package in Flutter/Dart projects and its generated files should not be included in coverage calculations.

Suggested change

bool _isGeneratedFile(String fileName) {

return fileName.endsWith('.g.dart') ||

fileName.endsWith('.pb.dart') ||

fileName.endsWith('.mocks.dart');

}

bool _isGeneratedFile(String fileName) {

return fileName.endsWith('.g.dart') ||

fileName.endsWith('.pb.dart') ||

fileName.endsWith('.mocks.dart') ||

fileName.endsWith('.freezed.dart');

}

I see no usages of https://pub.dev/packages/freezed but if reviewers think I should protect against this, I can go ahead and add it.

Same answer for addressing #11942 (comment)

reidbaker · 2026-06-24T14:27:13Z

+# command. If a package drops below its listed threshold, the check
+# will fail.
+custom_coverage_minimums:
+  camera_android: 30.0


omg this is so very low.

reidbaker · 2026-06-24T14:30:03Z

+        .childDirectory('script')
+        .childDirectory('configs')
+        .childFile('custom_coverage_minimums.yaml');
+    if (minimumsFile.existsSync()) {


Should we log that we are skipping coverage checks when the file does not exist? Or should we configure this command to fail if coverage checks are run and the file is missing then build in a yaml file that opts in camera_android and camera_android_camerax into this check?

Edit: I see below that you are using _customMinimums. My gut says that instead of building the list from where the files exist we should go with the pattern of having a yaml config file that opts our 2 packages in and then fails if the custom_coverage_minimums configs do not exist where we expect.

reidbaker · 2026-06-24T14:30:50Z

This file is making me rethink if the pre push skills should also support camera_android. That said we can add that functionality later if we want.

reidbaker · 2026-06-24T14:31:11Z

+
+  @override
+  final String description =
+      'Checks that code coverage meets the specified minimums '


Consider including where someone can find the specified minimums.

reidbaker · 2026-06-24T14:36:24Z

+    }
+    final double requiredCoverage = _customMinimums[packageName]!;
+
+    // Delete generated lcov.info.


Move this code. See comment https://github.com/flutter/packages/pull/11942/changes#r3467934044

reidbaker · 2026-06-24T14:37:26Z

+      return null;
+    }
+
+    return _calculateCoverage(lcovFile);


Suggested change

return _calculateCoverage(lcovFile);

calculatedCoverage = _calculateCoverage(lcovFile);

// Delete generated lcov.info.

final Directory coverageDir = package.directory.childDirectory('coverage');

if (coverageDir.existsSync()) {

coverageDir.deleteSync(recursive: true);

}

return calculatedCoverage;

reidbaker · 2026-06-24T14:43:08Z

+    final io.ProcessResult result = await processRunner.run(
+      'flutter',
+      <String>[
+      'test',


Non blocking nit. This means we are running all the tests twice. I dont want to block this pr on figuring out that efficiency unless you think we should but it should block adoption more broadly for the packages repo. If we did this efficiently the logic for opt-in package would live in the regular test command.

reidbaker · 2026-06-24T14:45:14Z

How does the coverage command work with the dart code defined in camera_android_camerax/.agents/skills/check-readiness/

reidbaker · 2026-06-24T14:46:38Z

+
+  /// Calculates code coverage for non-generated code files by finding
+  /// the percentage of covered lines of code over the total lines of code.
+  double _calculateCoverage(File lcovFile) {


I did not double check that this calculateCoverage function works as expected but you might consider making it visibleForTesting and adding unit tests just for the parsing.

reidbaker · 2026-06-24T14:47:19Z

+    }
+
+    if (linesFound == 0) {
+      return 100.0;


maybe this is correct but it feels weird to me to have lines found give 100% when there were no lines.

reidbaker · 2026-06-24T14:48:46Z

+  }
+}
+
+bool _isGeneratedFile(String fileName) {


This seems reasonable. FWIW the lcov command also has the ability to do exclusions.

first pass % tests

c01885c

flutter-dashboard Bot added the CICD Run CI/CD label Jun 18, 2026

camsim99 added 5 commits June 22, 2026 11:34

remove git stuff and make the test opt in

405bb67

quick self review

6a60b80

add tests

c573a96

address some holes in testing

144f4f3

mark bringup true

e77c789

github-actions Bot removed the CICD Run CI/CD label Jun 22, 2026

gemini-code-assist Bot reviewed Jun 23, 2026

View reviewed changes

defend against malformed yaml

146ab9b

flutter-dashboard Bot added the CICD Run CI/CD label Jun 23, 2026

camsim99 added CICD Run CI/CD and removed CICD Run CI/CD labels Jun 23, 2026

self review

56d8762

github-actions Bot removed the CICD Run CI/CD label Jun 23, 2026

camsim99 added the CICD Run CI/CD label Jun 23, 2026

camsim99 changed the title ~~[All] Add code coverage presubmit test~~ Add code coverage presubmit test Jun 23, 2026

camsim99 added CICD Run CI/CD and removed CICD Run CI/CD labels Jun 23, 2026

self review

fcdfb2c

github-actions Bot removed the CICD Run CI/CD label Jun 23, 2026

camsim99 marked this pull request as ready for review June 23, 2026 23:14

gemini-code-assist Bot reviewed Jun 23, 2026

View reviewed changes

camsim99 requested a review from reidbaker June 23, 2026 23:20

remove unrequired prop

6cc606c

flutter-dashboard Bot added the CICD Run CI/CD label Jun 23, 2026

reidbaker reviewed Jun 24, 2026

View reviewed changes

Uh oh!

Conversation

camsim99 commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-Review Checklist

Footnotes

Uh oh!

camsim99 commented Jun 22, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist Bot Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist Bot Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

camsim99 commented Jun 18, 2026 •

edited

Loading