Unit test with Set Environment Variables is running fine in my local but failing in Jenkins CI/CD build

[abitha90k]

In my unit test, I am setting the environment variables using stefanbirkner library. In local it is working fine. but when build it using Jenkins CI/CD pipeline, the build is getting failed with reasons corresponding to set environement.

Any info to resolve the issue is appreciated.

[stefanbirkner]

Can you please post the stacktrace of the failing test.

[abitha90k]

isUserAuthenticateSuccess(....xyz.XXXXTest) Time elapsed: 0.041 sec <<< ERROR!
14:12:20 java.lang.NullPointerException
14:12:20 at (code where the env variables has to be reset)
14:12:20 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
14:12:20 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
14:12:20 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
14:12:20 at java.lang.reflect.Method.invoke(Method.java:498)
14:12:20 at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:326)
14:12:20 at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie.java:88)
14:12:20 at org.junit.internal.runners.MethodRoadie.runBeforesThenTestThenAfters(MethodRoadie.java:96)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(PowerMockJUnit44RunnerDelegateImpl.java:310)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestInSuper(PowerMockJUnit47RunnerDelegateImpl.java:131)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.access$100(PowerMockJUnit47RunnerDelegateImpl.java:59)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner$TestExecutorStatement.evaluate(PowerMockJUnit47RunnerDelegateImpl.java:147)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.evaluateStatement(PowerMockJUnit47RunnerDelegateImpl.java:107)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(PowerMockJUnit47RunnerDelegateImpl.java:82)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThenTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:298)
14:12:20 at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie.java:86)
14:12:20 at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:49)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.invokeTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:218)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.java:160)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:134)
14:12:20 at org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:33)
14:12:20 at org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:45)
14:12:20 at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:136)
14:12:20 at org.powermock.modules.junit4.common.internal.impl.JUnit4TestSuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:121)
14:12:20 at org.powermock.modules.junit4.common.internal.impl.AbstractCommonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:57)
14:12:20 at org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner.java:59)
14:12:20 at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
14:12:20 at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
14:12:20 at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
14:12:20 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
14:12:20 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
14:12:20 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
14:12:20 at java.lang.reflect.Method.invoke(Method.java:498)
14:12:20 at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
14:12:20 at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
14:12:20 at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
14:12:20 at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
14:12:20 at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)****

[abitha90k]

I am unsure if the env variable is not setting or else the @PowerMockIgnore({"javax.crypto."}) is not being effective. I am using the @PowerMockIgnore({"javax.crypto."}) in my test.

[stefanbirkner]

The stacktrace contains no code of System Rules. Can you please show the test class.

[abitha90k]

Here is my piece of code, where the jenkin job fails

public String createSignature(String dateToSignNew, String saltToSignNew) throws Exception {
    final SecretKeySpec key = new SecretKeySpec(Shared_secret.getBytes(), HMAC_SHA1);
    final Mac mac = Mac.getInstance(HMAC_SHA1);
    mac.init(key);
    final byte[] hmacData = mac.doFinal(getStringToSign(dateToSignNew, saltToSignNew)
        .getBytes());
    return Base64.getEncoder().encodeToString(hmacData);
}

where Shared_secret = System.getenv("SHAREDSECRET");
Jenkins fails stating Shared_secret is null.

And my system rules in tests goes like this

@Rule
public static final EnvironmentVariables environmentVariables = new EnvironmentVariables();
public final ExpectedException exception = ExpectedException.none();

@BeforeClass
public static void beforeClass() {
    environmentVariables.set("sharedSecret", "YT7vFJyItIbI6ntkBnbvyd_i6");
    environmentVariables.set("SIGN_IN_SERVICE_URL", xyz.com);
    environmentVariables.set("SIGN_IN_QUERY_PARAM", abc);
     }

[stefanbirkner]

You have to use

@ClassRule
public static final EnvironmentVariables environmentVariables = new EnvironmentVariables();

instead of

@Rule
public static final EnvironmentVariables environmentVariables = new EnvironmentVariables();

With System Rules 1.18.0 you don't even need the method beforeClass because you can set the environment variables immediately

@ClassRule
public static final EnvironmentVariables environmentVariables = new EnvironmentVariables()
    .set("sharedSecret", "YT7vFJyItIbI6ntkBnbvyd_i6")
    .set("SIGN_IN_SERVICE_URL", xyz.com)
    .set("SIGN_IN_QUERY_PARAM", abc);

Please tell me whether the @ClassRule annotation fixes you problem.

[abitha90k]

Still the issue exists..

isUserAuthenticateFailure(com.expedia.csp.refundTracker.WMRServiceTest) Time elapsed: 0.011 sec <<< FAILURE!
15:20:49 junit.framework.ComparisonFailure: expected:<_JG-2LaEyFvJ6HE8EANunpBWgEDFE> but was:

The value _JG-2LaEyFvJ6HE8EANunpBWgEDFE is set as System variable

[stefanbirkner]

Can you show me the declaration of Shared_secret. Is it a static field that may get initialized before EnvironmentVariables can set the variable sharedSecret?

[abitha90k]

I tried both the ways. declaring directly while setting the environment variables also, set the variable as private static final String and referencing that variable while setting the environment

public class xyz {
    private static final String Shared_Secret= "zgdasgjnvadmgk+_ndjfnajksd";
}

also

environmentVariables.set("sharedSecret", "gdasgjnvadmgk+_ndjfnajksd");

both did not work.
I am using PowerMock Runner along with the rules.

[stefanbirkner]

May I rephrase my question. Do you have the following line in your code?

private static final String Shared_secret = System.getenv("SHAREDSECRET");

[abitha90k]

In my unit test, I am mocking (using powermock) the environment variable that is got from the system environment, for which I am using the system rules.

To answer your question, I have the below line in my code and in my unit test, I am mocking it with passing the actual value.

private static final String Shared_secret = System.getenv("SHAREDSECRET"); 

Now I am trying like this in my unit test

@ClassRule
public static final EnvironmentVariables environmentVariables = new EnvironmentVariables();
public final ExpectedException exception = ExpectedException.none();

@ClassRule
public static TestRule environmentVariableRule() {
    return new TestRule() {
        @Override
        public Statement apply(Statement statement, Description description) {
            environmentVariables.set("sharedSecret", "dasgjnvadmgk+_ndjfnajksd");
}

[stefanbirkner]

I think the problem is that Shared_secret is static. Therefore System.getenv("SHAREDSECRET") is only called once.

I have one possible explanation for your problem. Let class A be the class with the static field Shared_secret. On your local machine the class A is first use by your test. Therefore the EnvironmentVariables rule can set the environment variable SHAREDSECRET before class A reads the environment variable in order to initialize the static field. Therefore Shared_secret has the value dasgjnvadmgk+_ndjfnajksd. On your CI server the class A is first used by another test. Therefore the variable Shared_secret is initialized before the rule EnvironmentVariables sets the environment variable's value. This means that Shared_secret has the default value of the variable which is null.

If that assumption is correct then you can fix your test by changing the line

private static final String Shared_secret = System.getenv("SHAREDSECRET"); 

to

private final String Shared_secret = System.getenv("SHAREDSECRET"); 

Can you please try this out and tell me about the result. Please also revert the experiment about initializing the rule. If you use System Rules 1.18.0 your code should look like

@ClassRule
public static final EnvironmentVariables environmentVariables = new EnvironmentVariables()
    .set("sharedSecret", "dasgjnvadmgk+_ndjfnajksd");
@Rule
public final ExpectedException exception = ExpectedException.none();

[SwarupJami]

@stefanbirkner
I am encountering the similar issue. My Junit runs fine in IntelliJ when run individually but fails when executed thru the Test package .
As suggested I have used @ClassRule annotation and set the environment variable.
It would be difficult to change the code from
public final static String ENTITY = System.getenv("ENTITY")

to
public final String ENTITY = System.getenv("ENTITY")

as there are other dependent Static variables which define our business logic. All such static variables are defined in our Constants class, which is always used by every test.

Your last post in this thread clearly explains it(I have verified by printing the variable value before and after the environment variable overwrite), however could you please provide any other way to tackle this issue?

As a workaround, I have removed the final keyword and overriding its value in every JUnit using @Before annotation . All my tests are passing but this is not a cleaner solution as I have exposed the variable in the code .