Setting up a Jenkins build server on EC2

In my last post about setting up Jenkins I looked at how to do a basic Jenkins setup on an Ubuntu machine. In my case I set it up on an old machine which is fine when I’m working at home but if I make changes when I’m not at home or when my build machine isn’t running the changes are not built and tested. If you hadn’t worked it out, in a proper continuous integration environment you should be running your builds continuously. So this morning I set out to get an EC2 instance running Jenkins.

The other reason I wanted to have the build server running continually is that I need to start scheduling some jobs for Knowsis to do the NLP part of our process., which I could do with Cron, but buildservers liike Jenkins and Teamcity offer really flexible scheduling and a nice interface for feedback so I don’t need to worry about building one myself, for now.

Setting up an EC2 instance

The first step in the process is to set up your EC2 instance. Amazon kindly provide a free tier so you can get a free micro instance for a year. This should work for you initially if your builds aren’t overly complex.

I won’t run through exactly how to get your instance running as you can find plenty of guides online, if you are completely new to EC2 I would recommend this guide provided by Amazon.

One thing to note is that you should make sure you set up the security group for your image to allow all traffic on port 80 so you can actually see Jenkins.

Installing nginx

In my previous post I mentioned setting up nginx to route requests to Jenkins  but didn’t cover it. So we’ll go though it here as we need a webserver running to host the requests coming through.

We’ll need to use YUM here as apt-get and aptitude aren’t installed. Thankfully the Amazon package index includes a version of nginx.

yum install nginx

Once installed we should start the nginx server to make sure that we can see our new EC2 instance before proceeding.

sudo /etc/rc.d/init.d/nginx start

You should be be able to hit your instance in a web browser. You can get the public hostname of your instance from the AWS management console, but it should look something like this:

http://ec2-XX-XX-XX-XXX.compute-1.amazonaws.com/

Installing Jenkins

In my previous post we used aptitude to install Jenkins but the Amazon Linux AMI doesn’t have the aptitude package manager, s owe have to use YUM instead.

First we need to add the repository to the list of YUM repos:

sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo

and then get the GPG key:

sudo rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key

then we can install Jenkins:

yum install jenkins

The installer will install the server as well as create jenkins user under which the service will run. You can now start the jenkins service:

sudo /etc/init.d/jenkins start

As Jenkins runs on port 8080 by default, the next step is to get nginx to proxy all requests on port 80 to port 8080. You could probably just change Jenkins to run on port 80 by default if you wanted. Anyway, just change your nginx config (/etc/nginx/nginx.conf) so that the server section reads as follows:

server {
listen 80 default;
server_name _;
location /{
proxy_pass http://127.0.0.1:8080;
}
}

I won’t go into the details of nginx setup, but this is the minimum required to get you to a point of having Jenkins working.

Make sure you restart nginx to take account of the config changes,

sudo /etc/rc.d/init.d/nginx restart

You should be be able to view the jenkins homepage in a web browser using the same url as before:

http://ec2-XX-XX-XX-XXX.compute-1.amazonaws.com/

Security

One of the topics mentioned but not covered previously was security. As your build server is now visible publicly, you will want to set up some sort of security to prevent people from doing bad things.  The simplest way is to use Jenkins own user database, but there are other options to use an LDAP server or the underlying OS users. A few points to make sure that the server is secure:

  •  Disable the option to allow new users to sign up (unless you actually want people to be able to signup)
  • Change the authorisation section to either allow logged in users to do anything or use matrix based security and make sure anonymous users have no permissions

Setting up builds

You should refer back to my previous post on how to get your builds set up.7

Let me know how you get on

Advertisements

Music Beta by Google

It’s official – just announced now at Google IO.

As per all the rumours it allows you to upload your music via a desktop manager, then play in the browser, tablets and mobiles. It’s also able to create 25 track playlists based on audio data and not artist similarity. They are allowing you to upload up to 20,000 songs. Launch will be invitation only for US users. Free during beta.

But, there’s no music for sale….yet.


It’s Google Music … oh, no it’s Movies

The press has been awash with details of the Google music service since late last night. Predictions have been around the introduction of an unlicensed cloud locker service that allows users to upload 50GB of music for free. Just announced now at Google IO was something that no one had predicted, Movies.

Google have just announced a Google Movies service within the marketplace in the next few weeks that allows users to link movie rentals to their Google account meaning that they can rent anywhere and watch anywhere. Works online, on tablets and on mobiles.

Users will be able to “pin” movies to a device to allow them to watch offline when out of streaming range. Rentals will cost about $2 and be standard streaming rental terms of 30 days (reduced to 24 hours after first viewing commences).


Cloud Wars Episode II – The Swedes Strike Back

Back in April I correctly predicted that Spotify would soon offer their mobile service to all customers, including free accounts, following their restrictions placed on non paying users; this is a huge step for them in the ongoing battle to get a full cloud service out there. To be clear, this update does not allow non-paying users to stream music that they do not own – only the tracks they have synced from their personal computer to their device. The desktop version of Spotify has been able to sync a users owned music over WiFi to a the mobile device for some time, a smart move on Spotify’s part to save streaming bandwidth costs. This new update now includes iPod syncing, something that has required the owners to always be tied to iTunes. Today’s announcement is really significant in terms of getting users into the Spotify ecosystem and in my opinion really puts Lorentzon & Ek’s company out there as the leaders in the cloud race.

Apple, Amazon and Spotify already account for 90% of  worldwide digital music revenues for and I think this move could prove to be a significant shift in Spotify’s publicised revenue problems if they can get customers to move away from Apple. This is all the more likely given their pending US launch and the hype around the lack of launch for the last yeat. Having worked on the original purchasing implementation, which was powered by 7digital, it was actually quite interesting to see how many purchases were made through it when users could still listen for free – it really points to users’ preference to own their music over the “rental” streaming model.

I think the most interesting outcome from all of this will be the backlash reaction from Apple. In the past, Apple have changed the way that music is stored on iPods during software updates to prevent third-party software from being able to move music onto the device. It wouldn’t take much for Apple to do this and I wouldn’t be surprised if the next software updates will cause problems for the engineers  over in Gothenberg & Stockholm.

The other question surrounds the iOS app and whether Apple will allow future updates to it or if they will even just remove it from the app store entirely. At 7digital we’ve struggled to get our iOS app through the submission process; an app that does considerably less to compete with iTunes and the iPod compared to that from Spotify. Unless Spotify has an agreement with Apple – which I find highly unlikely, I wouldn’t be surprised if the app store guidelines change to say if your app name contains the word ‘Spot’  and your logo is green then it can’t exist in the app store.

On top of this there is the well documented case of Apple wanting 30% of any app subscriptions which would effectively ruin Spotify’s model as they will not be able to operate on these terms. I guess the ball is now in Apple’s court.