NoHttpResponseException between prnfb and bitbucket-slack-notifier

[mdecourcey]

We have been successfully using prnfb for many years to kick off builds in Jenkins, and those triggers continue to work.

I have followed the instructions on https://github.com/Igogrek/bitbucket-slack-notifier to integrate BitBucket with Slack using prnfb, bitbucket-slack-notifier and a custom bot in Slack. I have installed bitbucket-slack-notifier on the same server as bitbucket and verified that posting data to http://localhost:9999/notify or the ip address version of that successfully sends me a DM in Slack. But prnfb is getting a NoHttpResponseException in the logs whenever it tries POST to the same url. I've included the stack trace below, as well as configuration for both a jenkins button/notification that works and a slack button/notification that gets the exception.

I would appreciate any insight you might have. I'll continue to debug with my Operations team to see if we can figure this out from our side. The next thing I hope to try is putting b-s-n on a separate server, test that, and if that does not work give it a named domain that prnfb can call like it does with jenkins.rr.com.

• Plugin version used: 3.24

• Bitbucket Server version used: 5.8.1

• Testing against RequestBin I see the expected data sent.

• Testing on the bitbucket server and curling bitbucket-slack-notifier directly I get messages in Slack from my custom bot. I have also been able to successfully use Chrome Console to do the same from my local machine.

• Stack traces in Bitbucket Server log file:
  2018-12-10 11:21:16,664 ERROR [http-nio-7990-exec-216] miked @SNPZUZx679x88511135x2 1wn9e21 64.125.118.246,10.3.0.66 "POST /rest/prnfb-admin/1.0/settings/buttons/3c65bf38-6604-4837-8b4a-2489d8f9315c/press/repository/1193/pullrequest/55 HTTP/1.0" se.bjurr.prnfb.http.UrlInvoker org.apache.http.NoHttpResponseException: 10.3.0.66:9999 failed to respond at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) at se.bjurr.prnfb.http.UrlInvoker.doInvoke(UrlInvoker.java:376) at se.bjurr.prnfb.http.UrlInvoker.invoke(UrlInvoker.java:182) at se.bjurr.prnfb.listener.PrnfbPullRequestEventListener$1.invoke(PrnfbPullRequestEventListener.java:100) at se.bjurr.prnfb.listener.PrnfbPullRequestEventListener.notify(PrnfbPullRequestEventListener.java:324) at se.bjurr.prnfb.service.ButtonsService.doHandlePressed(ButtonsService.java:106) at se.bjurr.prnfb.service.ButtonsService.handlePressed(ButtonsService.java:158) at se.bjurr.prnfb.presentation.ButtonServlet.press(ButtonServlet.java:176) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.analytics.client.filter.UniversalAnalyticsFilter.doFilter(UniversalAnalyticsFilter.java:92) at com.atlassian.analytics.client.filter.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:39) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.plugin.connect.plugin.auth.scope.ApiScopingFilter.doFilter(ApiScopingFilter.java:81) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.stash.internal.spring.security.StashAuthenticationFilter.doFilter(StashAuthenticationFilter.java:85) at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doInsideSpringSecurityChain(BeforeLoginPluginAuthenticationFilter.java:112) at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:75) at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:94) at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.plugin.connect.plugin.auth.oauth2.DefaultSalAuthenticationFilter.doFilter(DefaultSalAuthenticationFilter.java:69) at com.atlassian.plugin.connect.plugin.auth.user.ThreeLeggedAuthFilter.doFilter(ThreeLeggedAuthFilter.java:109) at com.atlassian.jwt.internal.servlet.JwtAuthFilter.doFilter(JwtAuthFilter.java:32) at com.atlassian.analytics.client.filter.DefaultAnalyticsFilter.doFilter(DefaultAnalyticsFilter.java:38) at com.atlassian.analytics.client.filter.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:39) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doBeforeBeforeLoginFilters(BeforeLoginPluginAuthenticationFilter.java:90) at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:73) at com.atlassian.stash.internal.request.DefaultRequestManager.doAsRequest(DefaultRequestManager.java:89) at com.atlassian.stash.internal.hazelcast.ConfigurableWebFilter.doFilter(ConfigurableWebFilter.java:38) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.lang.Thread.run(Thread.java:745) ... 239 frames trimmed

• Configuration:

  • Global Settings:
    {"adminRestriction":"ADMIN","keyStorePassword":"KEEP_THIS_TO_LEAVE_UNCHANGED","keyStoreType":"","shouldAcceptAnyCertificate":false}
  • Buttons:
    [{"buttonFormList":[],"buttonFormListString":"[]","name":"Pipeline","projectKey":"RR","userLevel":"EVERYONE","uuid":"b276f976-e9d0-4066-82a4-aa6efda864e8"},{"buttonFormList":[],"buttonFormListString":"[]","name":"Slack","projectKey":"RR","userLevel":"EVERYONE","uuid":"9e6b6a9d-93c4-41ea-ab5d-82e2ecd5f3ee"}]
  • Notifications:
    [{"filterRegexp":"^Pipeline$","filterString":"${BUTTON_TRIGGER_TITLE}","headers":[],"injectionUrl":"https://jenkins.richrelevance.com/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,%22:%22,//crumb)","method":"GET","name":"Jenkins PR Build Button Trigger","password":"KEEP_THIS_TO_LEAVE_UNCHANGED","projectKey":"RR","proxyPassword":"KEEP_THIS_TO_LEAVE_UNCHANGED","proxyUser":"KEEP_THIS_TO_LEAVE_UNCHANGED","triggerIfCanMerge":"NOT_CONFLICTING","triggerIgnoreStateList":[],"triggers":["BUTTON_TRIGGER"],"updatePullRequestRefs":false,"url":"https://jenkins.richrelevance.com/job/root_pipeline/buildWithParameters?token=xxxxxxxxx&${EVERYTHING_URL}&BUILD_CAUSE=BUTTON_CLICKED&cause=PR+Build+Button+Clicked","user":"KEEP_THIS_TO_LEAVE_UNCHANGED","uuid":"1cf17a81-baf0-46da-a659-f8deaa8bd9e1","postContentEncoding":"NONE"},{"filterRegexp":"Slack","filterString":"${BUTTON_TRIGGER_TITLE}","headers":[],"method":"POST","name":"slack main","password":"KEEP_THIS_TO_LEAVE_UNCHANGED","postContent":"{\r\n \"pullRequestReviewers\": \"${PULL_REQUEST_REVIEWERS_UNAPPROVED_SLUG}\",\r\n \"pullRequestTitle\": \"${PULL_REQUEST_TITLE}\",\r\n \"pullRequestUrl\": \"${PULL_REQUEST_URL}\",\r\n \"pullRequestAuthorDisplayName\": \"${PULL_REQUEST_AUTHOR_DISPLAY_NAME}\",\r\n \"pullRequestAction\": \"${PULL_REQUEST_ACTION}\",\r\n \"buttonTrigger\": \"${BUTTON_TRIGGER_TITLE}\",\r\n \"pullRequestAuthor\": \"${PULL_REQUEST_AUTHOR_SLUG}\",\r\n \"pullRequestApprovedCount\": \"${PULL_REQUEST_REVIEWERS_APPROVED_COUNT}\",\r\n \"triggerUser\": \"${PULL_REQUEST_USER_SLUG}\",\r\n \"triggerDisplayName\": \"${PULL_REQUEST_USER_DISPLAY_NAME}\",\r\n \"comment\": \"${PULL_REQUEST_COMMENT_TEXT}\",\r\n \"commentAction\": \"${PULL_REQUEST_COMMENT_ACTION}\"\r\n}","proxyPassword":"KEEP_THIS_TO_LEAVE_UNCHANGED","proxyUser":"KEEP_THIS_TO_LEAVE_UNCHANGED","triggerIfCanMerge":"ALWAYS","triggerIgnoreStateList":[],"triggers":["APPROVED","COMMENTED","OPENED","BUTTON_TRIGGER","REOPENED"],"updatePullRequestRefs":false,"url":"http://10.3.0.66:9999/notify","user":"KEEP_THIS_TO_LEAVE_UNCHANGED","uuid":"59f6279e-6c96-45a0-9db4-67b0a796b291","postContentEncoding":"JSON","httpVersion":"HTTP_1_0"}]

[tomasbjerre]

What HTTP status code are you getting back when using curl?

Is the target system not being invoked? Or is it being invoked but still the plugin shows an exception?

[mdecourcey]

When using curl the HTTP status code is 200.

Using tcpdump I believe I have found the issue.

Short story: Content-type: application/json was not included as a header on the post when "Encode as JSON" was selected. Adding that as an "Optional HTTP headers to send with URL." fixed the issue and the notifications are now working.

Slightly longer story: Using tcpdump and recording the traffic to port 9999 on the server hosting bitbucket and the bitbucket-slack-notifier I was able to see the difference in packets between prnfb posting to b-s-n and curl/chrome posting to b-s-n. prnfb would not receive a response back AND it did not include all the same headers. It was missing Content-Type: application/json, Accept: application/json and Accept-Language: en-US, en. I found through trial and error that the only one of these needed to successfully call b-s-n was the Content-Type.

I'm not sure exactly what the intent behind "Encode as JSON" was, but my assumption was that it would add the Content-Type: application/json without adding anything to the optional headers. If that was the intent, then I believe this is a bug in prnfb. If it is not a bug, then I'll contact the maintainer of bitbucket-slack-notifier to update the integration instructions to include adding a the optional header.

[tomasbjerre]

Encoding as json just means variables will be encoded as json. So that they can safely be used in a post content json structure.

So not a bug!