Removing use of dart:mirrors to allow AOT compile

[davidmorgan]

The latest source_gen release deprecates TypeChecker.fromRuntime, because it uses mirrors

https://pub.dev/documentation/source_gen/3.1.0/source_gen/TypeChecker/TypeChecker.fromRuntime.html

the intended replacement is the new TypeChecker.typeNamed

https://pub.dev/documentation/source_gen/3.1.0/source_gen/TypeChecker/TypeChecker.typeNamed.html

--the is a little looser than TypeChecker.fromRuntime, it gets the name from the type you pass and then you should also explicitly pass a package name, so it matches name+package instead of full name+URI. This is usually what you want! As it doesn't make sense to stop detecting an annotation because it moved between files in the same package.

The next source_gen major version bump will remove TypeChecker.typeNamed, and then it will be possible to compile AOT. I need to teach build_runner to compile AOT in the meantime, work in progress :)

One important note, the source_gen GeneratedForAnnotation uses TypeChecker.fromRuntime, you can keep using the annotation generator but you should start passing the annotation package name now, it will switch to TypeChecker.typeNamed with the breaking release.

https://pub.dev/documentation/source_gen/3.1.0/source_gen/GeneratorForAnnotation/GeneratorForAnnotation.html

[lume-code]

Been watching out for this one for a while now. Congratulations on the completion :)

[davidmorgan]

Thanks! It was quite a journey ;)

Planning on doing a release for this refactor, then it's just a short hop from there to having AOT compile, hopefully ... :)

[lume-code]

I recently tried the AOT, and I thought it might be helpful to share the results:
JIT:

dart run build_runner build --delete-conflicting-outputs --force-jit
29s injectable_generator:injectable_builder on 526 inputs: 337 output, 189 no-op; spent 14s analyzing, 13s resolving, 2s building                                                                           
4s injectable_generator:injectable_config_builder on 1949 inputs: 1 same, 1948 no-op; spent 3s building                                                                                                     
10s graphql_codegen on 243 inputs: 243 same; spent 10s building                                                                                                                                             
8s freezed on 417 inputs: 207 same, 210 no-op; spent 5s analyzing, 3s building                                                                                                                              
0s flutter_gen_runner on 1 input: 1 same                                                                                                                                                                    
3s auto_route_generator on 218 inputs: 171 output, 47 no-op; spent 2s building, 1s analyzing                                                                                                                
0s auto_route_generator:auto_router_generator on 1 input: 1 same                                                                                                                                            
                                                                                                                                                                                                            
Built with build_runner/jit in 63s; wrote 961 outputs.

AOT:

18s injectable_generator:injectable_builder on 526 inputs: 337 output, 189 no-op; spent 8s analyzing, 8s resolving, 1s building                                                                             
4s injectable_generator:injectable_config_builder on 1949 inputs: 1 output, 1948 no-op; spent 3s building                                                                                                   
8s graphql_codegen on 243 inputs: 243 output; spent 7s building                                                                                                                                             
5s freezed on 417 inputs: 207 output, 210 no-op; spent 3s analyzing, 2s building                                                                                                                            
0s flutter_gen_runner on 1 input: 1 output                                                                                                                                                                  
2s auto_route_generator on 218 inputs: 171 output, 47 no-op; spent 1s building                                                                                                                              
0s auto_route_generator:auto_router_generator on 1 input: 1 output                                                                                                                                          
                                                                                                                                                                                                            
Built with build_runner/aot in 43s; wrote 961 outputs.

So roughly %32, yes (🎉).

However, I don't know if it is a characteristic of AOT or not, but from the moment the command is commenced until it actually starts building, it takes around 15 seconds. Just in AOT. JIT starts almost immediately.

[ykmnkmi]

Can AOT perform incremental compilation? As far as I remember, it always compiles everything from scratch.

PS. I mean build script itself.

[davidmorgan]

@lume-code Thanks!

That additional startup time is the AOT compile of the builders, it should usually only need to run once and be reused the next time. If you change your pubspec or "pub upgrade" it will need to run again. @ykmnkmi is correct that it runs from scratch, but actually JIT "compile" (to kernel) does too, it's rather that the AOT compile of the build script takes much longer. Then, when it's ready, it's faster :)

[gmpassos]

Here, the build time was reduced from 23s to 9s! 👏👏👏

---

dart run build_runner build -v

  21s reflection_factory on 124 inputs: 93 skipped, 31 same; spent 13s analyzing, 4s building, 3s resolving
  Built with build_runner/jit in 23s; wrote 31 outputs.

---

dart run build_runner build --force-aot -v

    9s reflection_factory on 124 inputs: 93 skipped, 31 same; spent 6s analyzing, 2s building
  Built with build_runner/aot in 9s; wrote 31 outputs.

[mraleph]

Could you measure total invocation time for the command (and do this repeated invocations) to include AOT (re)compilation costs? e.g.

time dart run build_runner build -v
time dart run build_runner build -v

time dart run build_runner build --force-aot -v
time dart run build_runner build --force-aot -v

[davidmorgan]

Thanks @gmpassos :) I also saw 21s->7s on one of the benchmarks, was very happy with that. AOT compiled builders are much better for startup.

@mraleph as you've guessed the AOT recompile is reused if nothing changed, it's a bit difficult to see what's happening as currently build_runner doesn't tell you that it's doing the AOT compile or how long it took. But I'm pretty sure the 9s AOT time is with a reused AOT compile as I think the compile takes 15-30s :)

For this release you can just pick a compile type with flags, but I think for the next release I will make it do something clever: on first build it will kick off both compiles in parallel and use the JIT one, then for subsequent builds it will use the AOT one.

[gmpassos]

For this release you can just pick a compile type with flags, but I think for the next release I will make it do something clever: on first build it will kick off both compiles in parallel and use the JIT one, then for subsequent builds it will use the AOT one.

Please add a flag to disable this, especially for CI environments (which are always fresh scenarios). In our CI check job, the process fails if the builder generates a new version of a generated file.

[davidmorgan]

Already done ;) the --force-jit flag does nothing today, it's there to force the current behavior for if you don't want what's coming up.

I'm curious, how does your CI check work? Specifically, how does it detect that some output changed? There is a feature request to make this easier ... ehm ... I don't think I ever filed an actual feature request, I've done that now #4253

[gmpassos]

I'm curious, how does your CI check work? Specifically, how does it detect that some output changed? There is a feature request to make this easier ... ehm ... I don't think I ever filed an actual feature request, I've done that now #4253

We use the package build_verify.
It only works if your code is in a git repository, ensuring that any change can be tracked safely.

---

Here’s the test code we use in our projects. It ignores any changes to pubspec.lock, preventing issues with dart pub upgrade in CI:

ensure_build_test.dart:

@TestOn('vm')
@Tags(['build', 'slow'])
@Timeout(Duration(minutes: 2))
import 'package:build_verify/build_verify.dart';
import 'package:test/test.dart';

void main() {
  test('ensure_build',
      () async => expectBuildClean(gitDiffPathArguments: [':!pubspec.lock']));
}