When using Jenkins to build iOS applications on Mac OS X, it’s very important to ensure the server is secure – both protected from unauthorized access, and having communications secured by SSL . When the server is not secure, confidential information can be exposed to the world via unencrypted HTTP. That is risky because anyone who can sniff network traffic between the Jenkins server and a client can observe everything, including user names, passwords, and other sensitive information that Jenkins may expose when viewed over the web. Even if you add username and password authentication, an attacker could sniff those over the wire unless connections to the server are protected with SSL encryption.
Why Secure It?
To secure Jenkins, you must use a valid SSL certificate. You can either buy a certificate from a traditional certificate authority, or use a free SSL certificate. When choosing either of these options, we need to evaluate carefully. There is a cost to the organization, both to register the certificates and for continued maintenance updates when they expire.
Buying a SSL will yield a certificate with 1 year or more validity with the ability to renew them. Most free SSL certificates come with 3 month validity and would require users to renew manually after that. Both are similar in maintenance costs but the cost of the paid certificates varies.
After comparing many available options, we chose Let’s Encrypt. It solved both the issue of getting a free certificate, and a good part of the maintenance burden, so we decided to try them.
Why Let’s Encrypt?
Let’s Encrypt is a new service powered by open source software that offers to encrypt your website’s SSL communications without charging for the certificates. It has these benefits:
- It’s free – Anyone who owns a domain name can use Let’s Encrypt to obtain a trusted certificate
- It’s trusted – Let’s Encrypt issues certificates trusted by most browsers and operating systems
- Lower Maintenance Burden – It’s easy to install and the certificates will renew automatically if set up correctly
Let’s Encrypt is run by a small team and relies on automation to keep costs down. That being the case, they are not able to offer direct support to subscribers. They do have some great support options through members of their community who do a great job of answering questions, and many of the most common questions have already been answered.
Let’s get started…
We wanted to ensure the Jenkins server will be protected with SSL encryption on its web interface by hooking up Apache as a reverse proxy and adding a LetsEncrypt SSL certificate to Apache. This Jenkins server ran Mac OS X, where Apache is also installed by default. We had access to the Mac OS X graphical console via remote control software. While the procedures outlined here are specific to Mac OS X, the similar procedures to secure can be applied to Linux or other UNIX-like operating systems.
Install Homebrew
Homebrew should be installed through a user admin account and not as root. Follow these commands to install Homebrew:
server:~ Client-AdminUser$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
server:~ Client-AdminUser$ brew install certbot
There are some dependencies that you should be aware of that you will need to install
- X code Tools:
- root # xcode-select –install
- Python pip:
- Follow the directions at https://pip.readthedocs.org/en/stable/installing/#install-pip
- Python virtualenv:
- Follow the directions at http://exponential.io/blog/2015/02/10/install-virtualenv-and-virtualenvwrapper-on-mac-os-x/
- Or you can use these commands to install pip:
$ sudo pip install virtualenv virtualenvwrapper
It seems like a lot to start, but just follow the links and it will be easy to install.
Install Let’s Encrypt
Let’s begin with installing Let’s encrypt on the terminal:
$ git clone https://github.com/letsencrypt/letsencrypt
After installation, check that your DocumentRoot directory is configured and present before performing the following actions. Use the following command to determine where the root directory is:
Example:
$ grep -r DocumentRoot /etc/apache2 /etc/apache2/extra/httpd-ssl.conf:DocumentRoot “/Library/WebServer/Documents” /etc/apache2/extra/httpd-ssl.conf~orig:DocumentRoot “/Library/WebServer/Documents” /etc/apache2/extra/httpd-ssl.conf~previous:DocumentRoot “/Library/WebServer/Documents” /etc/apache2/extra/httpd-vhosts.conf~orig: DocumentRoot “/usr/docs/dummy-host.example.com” /etc/apache2/extra/httpd-vhosts.conf~orig: DocumentRoot “/usr/docs/dummy-host2.example.com” /etc/apache2/extra/httpd-vhosts.conf~previous: DocumentRoot “/usr/docs/dummy-host.example.com” /etc/apache2/extra/httpd-vhosts.conf~previous: DocumentRoot “/usr/docs/dummy-host2.example.com” /etc/apache2/httpd.conf:# DocumentRoot: The directory out of which you will serve your /etc/apache2/httpd.conf:DocumentRoot “/Library/WebServer/Documents” /etc/apache2/httpd.conf: # access content that does not live under the DocumentRoot.
Now you see the root directory is at /Library/WebServer/Documents (example)
Change the directory to letsencrypt and run the certbot commands to perform the challenges and install the certificates. In the end, you will see the following:
[server:~] root # cd letsencrypt [server:~/letsencrypt] root# sudo ./certbot-auto certonly --webroot -w /Library/WebServer/Documents -d jenkins.example.com -d www.jenkins.example.com Saving debug log to /var/log/letsencrypt/letsencrypt.org Obtaining a new certificate Performing the following challenges: http-01 challenge for example.com http-01 challenge for www.example.com Using webroot path /Library/Webserver/Documents for all unmatched domains. Waiting for verification… Cleaning up challenges IMPORTANT NOTES: - Congratulations! Your certificate and chain have been saved at /etc/letsencrypt/live/domain.org/fullchain.pem. Your cert will expire on 2017-11-19. To obtain a new or tweaked version of this certificate in the future, simply run certbot-auto again. To non-interactively renew all of your certificates, run "certbot-auto renew" - If you like Certbot, please consider supporting our work by: Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate Donating to EFF: https://eff.org/donate-le
Your certificates will be created and saved in the following directories:
/etc/Letsencrypt/live/jenkins.example.com/cert.pem /etc/Letsencrypt/live/jenkins.example.com/chain.pem /etc/Letsencrypt/live/jenkins.example.com/fullchain.pem /etc/Letsencrypt/live/jenkins.example.com/privkey.pem
To install on Apache httpd-ssl.conf, you will only use 3 of these files: chain.pem, cert.pem, privkey.pem
To configure inside Apache2, use the following configuration files:
/etc/apache2/extra/httpd-ssl.conf /etc/apache2/extra/httpd-vhosts.conf
Before configuring these two config files we need to know a few things to secure the system. We are going to install Apache as a reverse proxy and run all the Jenkins requests through it, and have a transparent redirect from HTTP to HTTPS on both port 80 and on port 8081, where Jenkins will live.
Login to terminal as root.
Verify that httpd.conf is listening on port 80 and httpd-ssl.conf should listen on port 443.
Configure Apache as a reverse proxy and enable SSL
For these you need to configure httpd.conf (/etc/apache2/httpd.conf)
[server:~/apache2] root# vi httpd.conf
The config file should look as follows:
Listen *: 80
# Uncomment these lines for modules:
LoadModule headers_module modules/mod_headers.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule socache_shmcb_module modules/mod_socache_shmcb.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so</p>
ServerName localhost:80
# Deny access to the entirety of your server's filesystem. You must
# explicitly permit access to web content directories in other
# <Directory> blocks below.
<Directory />
Options Indexes FollowSymLinks MultiViews
AllowOverride none
Require all granted
</Directory>
Note: Require all should be granted.
Check the config files to ensure nothing is missing:
[DN312:/etc/apache2] root# apachectl configtest
Syntax OK
Voila! You have no errors.
Lastly, make sure to verify the snippet below is not commented out in the httpd.conf file in the “Supplemental configuration” section:
# Supplemental configuration # # The configuration files in the /private/etc/apache2/extra/ directory can be # included to add extra features or to modify the default configuration of # the server, or you may simply copy their contents here and change as # necessary. # Server-pool management (MPM specific) Include /private/etc/apache2/extra/httpd-mpm.conf # Fancy directory listings Include /private/etc/apache2/extra/httpd-autoindex.conf # Virtual hosts Include /private/etc/apache2/extra/httpd-vhosts.conf # Configure mod_proxy_html to understand HTML4/XHTML1 <IfModule proxy_html_module> Include /private/etc/apache2/extra/proxy-html.conf </IfModule> # Secure (SSL/TLS) connections Include /private/etc/apache2/extra/httpd-ssl.conf # # Note: The following must must be present to support # starting without SSL on platforms with no /dev/random equivalent # but a statically compiled-in mod_ssl. # <IfModule ssl_module> SSLRandomSeed startup builtin SSLRandomSeed connect builtin </IfModule> Include /private/etc/apache2/other/*.conf
That’s it with httpd.conf.
Moving onto the /etc/apache2/extra/httpd-ssl.conf file:
[server:~/apache2/extra] root# vi httpd-ssl.conf Listen *:443 <VirtualHost _default_:443> # General setup for the virtual host #DocumentRoot "/Library/WebServer/Documents" #ServerName jenkins.exanoke.com:443 #ServerAdmin webmaster@jenkins.example.com.com ErrorLog "/private/var/log/apache2/error_log" TransferLog "/private/var/log/apache2/access_log" # SSL Engine Switch: # Enable/Disable SSL for this virtual host. SSLEngine On
Ensure the lines are not commented out and are available.
Insert the certificates which are generated by giving the path of the Cert and should like these:
SSLCertificateFile "/private/etc/letsencrypt/live/example.com/cert.pem" #SSLCertificateFile "/private/etc/apache2/server-dsa.crt" #SSLCertificateFile "/private/etc/apache2/server-ecc.crt" # Server Private Key: # If the key is not combined with the certificate, use this # directive to point at the key file. Keep in mind that if # you've both a RSA and a DSA private key you can configure # both in parallel (to also allow the use of DSA ciphers, etc.) # ECC keys, when in use, can also be configured in parallel SSLCertificateKeyFile "/private/etc/letsencrypt/live/example.com/privkey.pem" #SSLCertificateKeyFile "/private/etc/apache2/server-dsa.key" #SSLCertificateKeyFile "/private/etc/apache2/server-ecc.key" # Server Certificate Chain: # Point SSLCertificateChainFile at a file containing the # concatenation of PEM encoded CA certificates which form the # certificate chain for the server certificate. Alternatively # the referenced file can be the same as SSLCertificateFile # when the CA certificates are directly appended to the server # certificate for convenience. SSLCertificateChainFile "/private/etc/letsencrypt/live/example.com/chain.pem"
You should insert the following lines after the certificates are given. It is the key concept allowing your apache to act as a reverse proxy and your server to be secure (https).
ProxyRequests Off ProxyPreserveHost On AllowEncodedSlashes NoDecode <Proxy *> Order deny,allow Allow from all </Proxy> ProxyPass / http://127.0.0.1:8081/ nocanon ProxyPassReverse / http://127.0.0.1:8081/ RequestHeader set X-Forwarded-Proto "https" RequestHeader set X-Forwarded-Port "443"
Now you should be able to configure apache as reverse proxy and your https URL will work fine with the newly generated certificates.
After changing apache as reverse proxy, you should change the Jenkins URL from:
http://example.com:8081 to https://example.com
Note: Make sure to configure it first or you will break the login to Jenkins
Redirect HTTP requests to HTTPS
After configuring httpd.conf and httpd-ssl.conf you will be able to login both http and https. You still need to ensure anyone hitting the HTTP URL from outside is redirected to the secure HTTPS page. For this, configure the httpd-vhost.conf file, which will redirect all the requests. Your config file should like this.
httpd-ssl.vhosts . (/etc/apache2/extra/httpd-ssl.conf)
<VirtualHost *:80>
ServerName jenkins.example.com:80
DocumentRoot "/Library/WebServer/Documents"
# Thanks Stack Overflow https://stackoverflow.com/a/27697308/424301
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</IfModule>
</VirtualHost>
<VirtualHost *:8081>
ServerName jenkins.example.com:8081
DocumentRoot "/Library/WebServer/Documents"
# Thanks Stack Overflow https://stackoverflow.com/a/27697308/424301
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://jenkins.example.com%{REQUEST_URI} [R=301,L]
</IfModule>
</VirtualHost>
The VirtualHost *:80 will redirect all the requests heading toward the standard HTTP port to the SSL port but you are still able to hit the URL having port number 8081, with the default Jenkins configuration. You should do some further securing of Jenkins, so that it doesn’t listen on port 8081 anymore. Then you can redirect users hitting the HTTP port 8081 URL they formerly used to the HTTPS server.
To do this, we have to:
- Stop Jenkins listening on port 8081
- Make it listen to localhost only
- Have Apache listen on port 8081 on just the public IP address.
Now you are able to access the https URL even if you try to hit http with port 80 or port 8081.
Stop Jenkins from listening on port 8081:
The final task will be getting Jenkins to stop listening on port 8081. This is possible by making changes in the plist file.
What is plist?
Property list files are often used to store a user’s settings. They are also used to store information about bundles and applications.
You can locate your plist file under /Library/Preferences/org.jenkins-ci.plist
XCode (installed earlier) has a plist editor built in. Login as root and open the plist file, using the following command:
[server:~ /Library/Preferences] root# sudo open org.jenkins-ci.plist
If you installed Xcode tools correctly it will open an application that looks like this:
Add a new row and select type as String. As shown in the image, enter the values for httpListenAddress.
Relaunch the Jenkins daemon so that it reads the file, picks up the edited values, and follows the commands:
launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist
These commands will stop Jenkins from listening on port 8081 on all interfaces and it will start listening only on localhost.
Open the config file httpd.conf and write a host condition. You should add a Listen address after the port 80 clause, replacing 10.11.13.13 with your public IP address:
#Listen 12.34.56.78:80 Listen *:80 Listen 10.11.12.13:8081 # Dynamic Shared Object (DSO) Support
This condition will make Apache start listening on port 8081 on the public IP address.
Here is a list of commands you should be familiar with while doing these processes:
# Command used to check the Listen address on server.
sudo netstat -nap tcp | grep LISTEN
# To restart apache
apachectl restart or httpd restart
# Check configuration of files httpd.conf and httpd-ssl.conf
apachectl configtest
# Check server is running.
ps -ef | grep java
# To ping the url on server, So that you can know response of the page.
curl -v http://example.com:8081
Automating Certificates
Let’s Encrypt’s Certbot program can be configured to renew your certificates automatically before they expire. Since Let’s Encrypt certificates last for 90 days, you can test automatic renewal for your certificates by running this command or by setting up a cron job.
[server:~/letsencrypt] root# ./certbot-auto certonly --webroot --webroot-path /Users/Your-Site-User-Account/Sites/ --email webmaster@domain.org -d example.com -d www.example.com
Setting up cron job using crontab
To run the renewal check daily, we will use cron, a standard system service for running periodic jobs. We tell cron what to do by opening and editing a file called crontab.
[server:~/apache2] root# sudo crontab -e
This opens vi editor for you. Create the cron command using the following graphical syntax.
Your text editor will open the default crontab which is a text file with some help text in it.
An example command:
. . …………### 25 3 * * * /usr/bin/certbot renew --quiet
would be at the end of the file. Then save and close it.
The 25 3 * * * part of this line means “run the following command at 3:25 am, every day”. You may choose any time.
The renew command for Certbot will check all certificates installed on the system and update any that are set to expire in less than thirty days. --quiet tells Certbot not to output information or wait for user input.
All installed certificates will be automatically renewed and reloaded when they have thirty days or less before they expire by using cron.
Conclusion
By following this step by step procedure, you can get your certificates installed and configured in Apache or Nginx, and have Jenkins proxied in a secure way, while keeping existing URLs working through redirects. I highly recommend trying to use Let’s Encrypt certificates. If you follow this procedure, you can get the server responding on HTTPS on both example.com and www.example.com and a green lock in the browser will appear to indicate secure status.
