Add Baseline Profile Module (#928)

* feat: add baseline profile

* feat: minor changes

* Update generate-baseline-profile.yml

* Update generate-baseline-profile.yml

* Cleanup and generate baseline profile

* feat(build): Add workflow to generate baseline profile

* Install GMD image for baseline profile generation
* Accept Android licenses
* Generate baseline profile for free release variant
* Upload generated baseline profiles as artifact

* Fix gmd source

* feat: Disable 64-bit requirement for Pixel 6 API 34

The 64-bit requirement has been disabled for the "pixel6Api34" managed virtual device.

* feat: Configure and run baseline profile generation

*   Cleared the default managed devices in `baselineprofile/build.gradle.kts` and explicitly added "pixel6Api34".
* Configured the GitHub Actions workflow `generate-baseline-profile.yml` to setup the managed device pixel6Api34.
* Configured the workflow to build all build type and flavor permutations.
* Added the option to show kernel logging and use "swiftshader_indirect" for GPU in the tests.

* Refactor: Update baseline profile setup task

*   Changed the Gradle task from `:benchmarks:pixel6Api34Setup` to `:baselineprofile:pixel6Api34Setup` in the GitHub Actions workflow for setting up the baseline profile generation.

* CI: Comment out GMD image installation in baseline profile generation workflow

The GMD image installation step in the `generate-baseline-profile.yml` workflow has been commented out.

* feat: Add Baseline Profile generation

*   The Baseline Profile generation is added in the project
*   Update baselineprofile module to use connected devices for test execution.
*   Create a new shell script `generateBaselineProfile.sh` to install app and baseline profile and pull file to `baseline-prof.txt`
*   Modify `generate-baseline-profile.yml` github workflow to generate profile on device using `reactivecircus/android-emulator-runner` action
*   Fix the package name to `com.yogeshpaliyal.keypass` in `BaselineProfileGenerator.kt` to pull the data.

* CI: Update Baseline Profile device to Nexus 6

* Changed the device profile used for Baseline Profile generation from Pixel 6 to Nexus 6 in the GitHub Actions workflow.

Author: yogeshpaliyal

.github/workflows/generate-baseline-profile.yml

@@ -11,21 +11,59 @@ jobs:
 
     - name: Setting up project
       uses: ./.github/actions/setup
-      
-    - name: Install GMD image for baseline profile generation
-      run: yes | "$ANDROID_HOME"/cmdline-tools/latest/bin/sdkmanager "system-images;android-34;aosp_atd;x86_64"
-  
+
+#
+#    - name: Install GMD image for baseline profile generation
+#      run: yes | "$ANDROID_HOME"/cmdline-tools/latest/bin/sdkmanager "system-images;android-34;aosp_atd;x86_64"
+
     - name: Accept Android licenses
       run: yes | "$ANDROID_HOME"/cmdline-tools/latest/bin/sdkmanager --licenses || true
-  
-    - name: Generate profile
-      run: ./gradlew :app:generateProReleaseBaselineProfile
-           -Pandroid.testoptions.manageddevices.emulator.gpu="swiftshader_indirect"
 
-     # Upload the entire generated folder and attach it to the CI run
+        # Now use reactivecircus/android-emulator-runner to spin up an emulator and run our
+        # baseline profile generator. We need to manually pull the baseline profiles off the
+        # emulator when using the GA runner
+    - name: Run benchmark
+      uses: reactivecircus/android-emulator-runner@v2
+      with:
+        api-level: 34
+        target: google_apis
+        arch: x86_64
+        profile: Nexus 6
+        script: |
+          ./gradlew :app:generateFreeReleaseBaselineProfile
+#
+#    - name: Setup GMD
+#      run: ./gradlew :baselineprofile:pixel6Api34Setup
+#        --info
+#        -Pandroid.experimental.testOptions.managedDevices.emulator.showKernelLogging=true
+#        -Pandroid.testoptions.manageddevices.emulator.gpu="swiftshader_indirect"
+#
+#    - name: Build all build type and flavor permutations including baseline profiles
+#      run: ./gradlew :app:assemble
+#          -Pandroid.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=baselineprofile
+#          -Pandroid.testoptions.manageddevices.emulator.gpu="swiftshader_indirect"
+#          -Pandroid.experimental.testOptions.managedDevices.emulator.showKernelLogging=true
+#
+
+#    - name: Generate profile
+#      run: ./gradlew :app:generateFreeReleaseBaselineProfile
+#        -Pandroid.testoptions.manageddevices.emulator.gpu="swiftshader_indirect"
+
+        # Now use reactivecircus/android-emulator-runner to spin up an emulator and run our
+        # baseline profile generator. We need to manually pull the baseline profiles off the
+        # emulator when using the GA runner
+#    - name: Run benchmark
+#      id: build
+#      run: |
+#         # Run our benchmark, enabling only tests using BaselineProfile
+#         ./gradlew pixel6Api34ProBenchmarkReleaseAndroidTest -Pandroid.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile
+#         # Need to manually pull the generated profiles from the emulator
+#         adb pull /sdcard/Android/media/app.tivi.benchmark benchmark/build/outputs/baseline-prof/
+
+      # Upload the entire generated folder and attach it to the CI run
     - name: Attach baseline profile
       uses: actions/upload-artifact@v4
       with:
         name: Baseline profile output
-        path: benchmark/build/outputs/baseline-prof
+        path: app/src/freeRelease/generated/baselineProfiles/
 

.gitignore

@@ -1,4 +1,5 @@
 .gradle
+.kotlin
 local.properties
 build/
 !gradle/wrapper/gradle-wrapper.jar
@@ -45,4 +46,4 @@ bin/
 .vscode/
 
 ### Mac OS ###
-.DS_Store
+.DS_Store

.java-version

@@ -1 +1 @@
-openjdk64-17.0.7
+17

app/build.gradle.kts

@@ -7,6 +7,7 @@ plugins {
     id("com.google.dagger.hilt.android")
     id("org.jetbrains.kotlin.plugin.serialization")
     id("org.jetbrains.kotlin.plugin.compose")
+    id("androidx.baselineprofile")
 }
 
 val appPackageId = "com.yogeshpaliyal.keypass"
@@ -103,13 +104,15 @@ dependencies {
 
     // api(project(":shared"))
     api(project(":common"))
+    implementation("androidx.profileinstaller:profileinstaller:1.3.1")
     testImplementation("junit:junit:4.13.2")
     androidTestImplementation("androidx.test.ext:junit:1.2.1")
     androidTestImplementation("androidx.test.espresso:espresso-core:3.6.1")
     androidTestImplementation("androidx.test:core-ktx:1.6.1")
     androidTestImplementation("androidx.test.ext:junit-ktx:1.2.1")
     // Test rules and transitive dependencies:
     androidTestImplementation("androidx.compose.ui:ui-test-junit4:${Versions.compose}")
+    "baselineProfile"(project(":baselineprofile"))
     // Needed for createAndroidComposeRule, but not createComposeRule:
     debugImplementation("androidx.compose.ui:ui-test-manifest:${Versions.compose}")
     androidTestImplementation("androidx.compose.ui:ui-test-junit4:${Versions.compose}")

baselineprofile/.gitignore

@@ -0,0 +1 @@
+/build

baselineprofile/build.gradle.kts

@@ -0,0 +1,74 @@
+import com.android.build.api.dsl.ManagedVirtualDevice
+
+plugins {
+    id("com.android.test")
+    id("org.jetbrains.kotlin.android")
+    id("androidx.baselineprofile")
+}
+
+android {
+    namespace = "com.yogeshpaliyal.baselineprofile"
+    compileSdk = 34
+
+    compileOptions {
+        sourceCompatibility = JavaVersion.VERSION_1_8
+        targetCompatibility = JavaVersion.VERSION_1_8
+    }
+
+    kotlinOptions {
+        jvmTarget = "1.8"
+    }
+
+    defaultConfig {
+        minSdk = 28
+        targetSdk = 34
+
+        testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
+    }
+
+    targetProjectPath = ":app"
+
+    flavorDimensions += listOf("default")
+    productFlavors {
+        create("free") { dimension = "default" }
+        create("pro") { dimension = "default" }
+    }
+
+    // This code creates the gradle managed device used to generate baseline profiles.
+    // To use GMD please invoke generation through the command line:
+    // ./gradlew :app:generateBaselineProfile
+    testOptions.managedDevices.devices {
+        create<ManagedVirtualDevice>("pixel6Api34") {
+            device = "Pixel 6"
+            apiLevel = 34
+            require64Bit = false
+            systemImageSource = "aosp-atd"
+        }
+    }
+}
+
+// This is the configuration block for the Baseline Profile plugin.
+// You can specify to run the generators on a managed devices or connected devices.
+baselineProfile {
+    // This specifies the managed devices to use that you run the tests on.
+//    managedDevices.clear()
+//    managedDevices += "pixel6Api34"
+    useConnectedDevices = true
+}
+
+dependencies {
+    implementation("androidx.test.ext:junit:1.2.1")
+    implementation("androidx.test.espresso:espresso-core:3.6.1")
+    implementation("androidx.test.uiautomator:uiautomator:2.3.0")
+    implementation("androidx.benchmark:benchmark-macro-junit4:1.2.4")
+}
+
+androidComponents {
+    onVariants { v ->
+        val artifactsLoader = v.artifacts.getBuiltArtifactsLoader()
+        v.instrumentationRunnerArguments.put(
+            "targetAppId",
+            v.testedApks.map { artifactsLoader.load(it)?.applicationId }
+        )
+    }
+}

baselineprofile/src/main/AndroidManifest.xml

@@ -0,0 +1 @@
+<manifest />

baselineprofile/src/main/java/com/techpaliyal/baselineprofile/BaselineProfileGenerator.kt

@@ -0,0 +1,67 @@
+package com.techpaliyal.baselineprofile
+
+import androidx.benchmark.macro.junit4.BaselineProfileRule
+import androidx.test.ext.junit.runners.AndroidJUnit4
+import androidx.test.filters.LargeTest
+import androidx.test.platform.app.InstrumentationRegistry
+import org.junit.Rule
+import org.junit.Test
+import org.junit.runner.RunWith
+
+/**
+ * This test class generates a basic startup baseline profile for the target package.
+ *
+ * We recommend you start with this but add important user flows to the profile to improve their performance.
+ * Refer to the [baseline profile documentation](https://d.android.com/topic/performance/baselineprofiles)
+ * for more information.
+ *
+ * You can run the generator with the "Generate Baseline Profile" run configuration in Android Studio or
+ * the equivalent `generateBaselineProfile` gradle task:
+ * ```
+ * ./gradlew :app:generateReleaseBaselineProfile
+ * ```
+ * The run configuration runs the Gradle task and applies filtering to run only the generators.
+ *
+ * Check [documentation](https://d.android.com/topic/performance/benchmarking/macrobenchmark-instrumentation-args)
+ * for more information about available instrumentation arguments.
+ *
+ * After you run the generator, you can verify the improvements running the [StartupBenchmarks] benchmark.
+ *
+ * When using this class to generate a baseline profile, only API 33+ or rooted API 28+ are supported.
+ *
+ * The minimum required version of androidx.benchmark to generate a baseline profile is 1.2.0.
+ **/
+@RunWith(AndroidJUnit4::class)
+@LargeTest
+class BaselineProfileGenerator {
+
+    @get:Rule
+    val rule = BaselineProfileRule()
+
+    @Test
+    fun generate() {
+        // The application id for the running build variant is read from the instrumentation arguments.
+        rule.collect(
+            packageName = "com.yogeshpaliyal.keypass",
+
+            // See: https://d.android.com/topic/performance/baselineprofiles/dex-layout-optimizations
+            includeInStartupProfile = true
+        ) {
+            // This block defines the app's critical user journey. Here we are interested in
+            // optimizing for app startup. But you can also navigate and scroll through your most important UI.
+
+            // Start default activity for your app
+            pressHome()
+            startActivityAndWait()
+
+            // TODO Write more interactions to optimize advanced journeys of your app.
+            // For example:
+            // 1. Wait until the content is asynchronously loaded
+            // 2. Scroll the feed content
+            // 3. Navigate to detail screen
+
+            // Check UiAutomator documentation for more information how to interact with the app.
+            // https://d.android.com/training/testing/other-components/ui-automator
+        }
+    }
+}

baselineprofile/src/main/java/com/techpaliyal/baselineprofile/StartupBenchmarks.kt

@@ -0,0 +1,76 @@
+package com.techpaliyal.baselineprofile
+
+import androidx.benchmark.macro.BaselineProfileMode
+import androidx.benchmark.macro.CompilationMode
+import androidx.benchmark.macro.StartupMode
+import androidx.benchmark.macro.StartupTimingMetric
+import androidx.benchmark.macro.junit4.MacrobenchmarkRule
+import androidx.test.ext.junit.runners.AndroidJUnit4
+import androidx.test.filters.LargeTest
+import androidx.test.platform.app.InstrumentationRegistry
+import org.junit.Rule
+import org.junit.Test
+import org.junit.runner.RunWith
+
+/**
+ * This test class benchmarks the speed of app startup.
+ * Run this benchmark to verify how effective a Baseline Profile is.
+ * It does this by comparing [CompilationMode.None], which represents the app with no Baseline
+ * Profiles optimizations, and [CompilationMode.Partial], which uses Baseline Profiles.
+ *
+ * Run this benchmark to see startup measurements and captured system traces for verifying
+ * the effectiveness of your Baseline Profiles. You can run it directly from Android
+ * Studio as an instrumentation test, or run all benchmarks for a variant, for example benchmarkRelease,
+ * with this Gradle task:
+ * ```
+ * ./gradlew :baselineprofile:connectedBenchmarkReleaseAndroidTest
+ * ```
+ *
+ * You should run the benchmarks on a physical device, not an Android emulator, because the
+ * emulator doesn't represent real world performance and shares system resources with its host.
+ *
+ * For more information, see the [Macrobenchmark documentation](https://d.android.com/macrobenchmark#create-macrobenchmark)
+ * and the [instrumentation arguments documentation](https://d.android.com/topic/performance/benchmarking/macrobenchmark-instrumentation-args).
+ **/
+@RunWith(AndroidJUnit4::class)
+@LargeTest
+class StartupBenchmarks {
+
+    @get:Rule
+    val rule = MacrobenchmarkRule()
+
+    @Test
+    fun startupCompilationNone() =
+        benchmark(CompilationMode.None())
+
+    @Test
+    fun startupCompilationBaselineProfiles() =
+        benchmark(CompilationMode.Partial(BaselineProfileMode.Require))
+
+    private fun benchmark(compilationMode: CompilationMode) {
+        // The application id for the running build variant is read from the instrumentation arguments.
+        rule.measureRepeated(
+            packageName = InstrumentationRegistry.getArguments().getString("targetAppId")
+                ?: throw Exception("targetAppId not passed as instrumentation runner arg"),
+            metrics = listOf(StartupTimingMetric()),
+            compilationMode = compilationMode,
+            startupMode = StartupMode.COLD,
+            iterations = 10,
+            setupBlock = {
+                pressHome()
+            },
+            measureBlock = {
+                startActivityAndWait()
+
+                // TODO Add interactions to wait for when your app is fully drawn.
+                // The app is fully drawn when Activity.reportFullyDrawn is called.
+                // For Jetpack Compose, you can use ReportDrawn, ReportDrawnWhen and ReportDrawnAfter
+                // from the AndroidX Activity library.
+
+                // Check the UiAutomator documentation for more information on how to
+                // interact with the app.
+                // https://d.android.com/training/testing/other-components/ui-automator
+            }
+        )
+    }
+}

build.gradle.kts

@@ -28,6 +28,8 @@ plugins {
     id("com.google.dagger.hilt.android") version ("2.56.1") apply false
     id("com.gradle.enterprise") version("3.19.2") apply false
     id("org.jetbrains.kotlin.plugin.serialization") version (Versions.kotlin)
+    id("com.android.test") version "8.5.1" apply false
+    id("androidx.baselineprofile") version "1.2.3" apply false
 }
 
 

gradle.properties

@@ -17,15 +17,10 @@ org.gradle.parallel=true
 android.useAndroidX=true
 # Kotlin code style for this project: "official" or "obsolete":
 kotlin.code.style=official
-
 android.enableR8.fullMode=true
-
-
 #kotlin.native.enableDependencyPropagation=false
 android.defaults.buildfeatures.buildconfig=true
 android.nonTransitiveRClass=false
 android.nonFinalResIds=false
-
 kotlin.mpp.androidGradlePluginCompatibility.nowarn=true
-
-
+org.gradle.configuration-cache=true

scripts/generateBaselineProfile.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+./gradlew :app:installFreeBenchmarkRelease
+
+./gradlew  :baselineprofile:installFreeBenchmarkRelease
+
+./gradlew :baselineprofile:connectedFreeBenchmarkReleaseAndroidTest -Pandroid.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile
+adb pull /sdcard/Android/media/com.yogeshpaliyal.baselineprofile  app/src/main/baseline-prof.txt

settings.gradle.kts

@@ -28,4 +28,5 @@ rootProject.name = "KeyPass"
 include(":app")
 include(":shared")
 include(":common")
-//include(":desktop")
+//include(":desktop")
+include(":baselineprofile")