bookmark_borderSetting up Jenkins with ReverseProxy and with with github-webhook

Recently I wanted to set up Jenkins and have it work with Github webhooks. It went fairly smoothly, but I ran into some edge cases which I wanted to elaborate on:

In this case there is a dedicated Debian server, using Apache as webserver. It is using Letsencrypt and runs several apps with their own subdomains.

One of those applications is Jenkins which is available through ci.<hostname>.com.

In order to achieve I basically followed the instructions at Jenkins.

In this case, Jenkins runs (configure it at /etc/default/jenkins) at localhost:8102 without context (ie /). Then, in Apache we configure this as so:

  ProxyPass / http://localhost:8102/ nocanon
  ProxyPassReverse / http://localhost:8102/

Make sure the trialing slash is there. But, still, this is standard stuff if you read the instructions at Jenkins.

Then, I still ran into problems using the github-webhook. I tested first using Postman: I send a POST to https://ci.<domain>.com/github-webhook and to my surprise it said:

java.lang.Exception: Method POST required
	at org.kohsuke.stapler.HttpResponses.error(HttpResponses.java:83)
	at org.jenkinsci.plugins.github.webhook.RequirePostWithGHHookPayload$Processor.shouldBePostMethod(RequirePostWithGHHookPayload.java:92)

Consulting apache logs results into:

"POST /github-webhook HTTP/1.1" 302 3613 "-" "PostmanRuntime/7.26.1"
"GET /github-webhook/ HTTP/1.1" 405 7097 "https://ci.<domain>.com/github-webhook" "PostmanRuntime/7.26.1"

This is weird, because I can login at Jenkins (which is using a POST) so it shouldn’t be possible that all POST requests become GET. Something else is wrong?

One way to test it is to just use curl on the machine itself. First, lets eliminate Apache’s role and see if we can access the url directly. Ie, since on the machine jenkins runs on port 8102 we should be able to access it like so:

curl -H "Content-Type: application/json" -i -X POST http://localhost:8102/github-webhook

and the response…

$ curl -H "Content-Type: application/json" -i -X POST http://localhost:8102/github-webhook
HTTP/1.1 302 Found
Date: Thu, 30 Jul 2020 10:09:36 GMT
X-Content-Type-Options: nosniff
Location: http://localhost:8102/github-webhook/
Content-Length: 0

Ah! so it looks like Jenkins is giving us a 302.

So what gives?

After a while I figured the url needs a trailing slash (/facepalm), and sadly I figured it after searching the web for a while. I hope this post makes it easier for people to figure this out.

So, at the end the url should become /github-webhook/. Now, testing again with curl:

$ curl -H "Content-Type: application/json" -i -X POST http://localhost:8102/github-webhook/
HTTP/1.1 400 Hook should contain event type
Date: Thu, 30 Jul 2020 10:09:00 GMT
X-Content-Type-Options: nosniff
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/html;charset=iso-8859-1
Content-Length: 372

And so we got a 400 now. This is good.

Retesting with Postman gave the same results.

Once setup in Github webhooks, it also worked fine!

So, remember, make sure a trailing slash is at that github webook url! –> /github-webhook/