Dec 10

Crickscore was originally created to allow me to try out new technologies in a place that didn’t really matter but was still important enough that it actually had to work and would be a benefit to people.

So Crickscore evolved, and now is used full time by 2 teams with an expectation that after a few more features are added it will be opened to the general public.

Over the last couple of days, one of the teams had a photo to post to the site, which was easily placed in the tomcat directory and linked to via code in the game description.  But as soon as I was to re-deploy the application the image would be gone.

I needed to build a longer term storage system for these items, allowing for  images to be uploaded and also linked to.

This saw the creation of the Media section.  As a registered user who is assigned to a team you can now upload image files to be stored and served from the crickscore system.

Its as simple as that.  You can also just link to an image already publically available on the web by selecting the Link option.

At the moment these item’s just appear on the Media tab for the team, but future enhancements will allow people and individual games to be linked to the images.

Also there is a hope that some basic video formats will be supported, but don’t hold your breath.  That is probably better done by uploading to YouTube and linking the document back.

So there’s the new feature, but the fuss is actually on how its done, if you are inclined read on to see how this was done, and proof of how quick and flexible this new “Cloud based services” era really is.

Geeky Stuff

We’ve been looking at internet hosted services (see I didn’t say Cloud) for a few things recently, most of which focused around Amazons new AWS services.

I have been playing with S3 to upload backups of my servers to the Cloud, as well as attempting to backup all the photos on the home PC.

When I saw a need for Crickscore to handle images, I thought that this would be a good time to try out serving these parts from Amazon’s Cloud Front.

For those that haven’t heard, Amazon have just released a CDN that fronts off your S3 buckets to allow any resources stored in the bucket to be served. Read a much better description over a the official Cloud Front site.

What I wanted to do was take any media file that was uploaded to crickscore and pass it through to S3 so that I could link to it using CloudFront.

I think there are ways I can directly post items up to S3, but I want to check the size, and content first and even do so processing on it, like resizing or watermarking first before it becomes publically available.

The first thing I needed to do was to find a Java Library that would allow me to administer s3 buckets, and once I found this library there wasn’t much more.

I’m using the jets3t library, which pretty much does all the hard work.  My application framework (WAF) handles multipart form uploads, so I end up with a byte[] which I can then just push up to s3 and bang its done.

Really, it’s that simple.

Code Sample

Here’s the code that I used, taken almost directly from the examples in the jets3t library.

public static Media uploadContent(User user, String name, Team team, byte[] content, String contentType) throws S3ServiceException, MediaException {

        String awsAccessKey = SystemSettings.getValue(SystemSettingConstants.S3_ACCESSKEY);
        String awsSecretKey = SystemSettings.getValue(SystemSettingConstants.S3_SECRETKEY);
        String s3BucketName = SystemSettings.getValue(SystemSettingConstants.S3_BUCKET_NAME);
        String urlPrefix = SystemSettings.getValue(SystemSettingConstants.S3_URL_PREFIX);
        //http://content.fixturetime.com/

        AWSCredentials awsCredentials =  new AWSCredentials(awsAccessKey, awsSecretKey);

        S3Service s3Service = new RestS3Service(awsCredentials);

        S3Bucket contentBucket = s3Service.getBucket(s3BucketName);

        if (contentBucket == null) {
            contentBucket = s3Service.createBucket(s3BucketName);
        }

        validateContentType(contentType);

        String filename = null;
        while (filename == null) {
            filename = generateFileName(team, contentType); 
              if (doesFileExist(s3Service, contentBucket, filename)) { 
                  filename = null; 
              }
        }

        // Create an object containing a greeting string as input stream data.
        S3Object imageFile = new S3Object(filename);
        ByteArrayInputStream greetingIS = new ByteArrayInputStream(content);
        imageFile.setDataInputStream(greetingIS);
        imageFile.setContentLength(greetingIS.available());
        imageFile.setContentType(contentType);
        imageFile.setAcl(AccessControlList.REST_CANNED_PUBLIC_READ);

        s3Service.putObject(contentBucket, imageFile);
        Media m = Media.create(user, team, name, contentType, urlPrefix + filename);

        return m;
    }

This function is pretty self explanatory, but I’ll go through it here for completeness.

You’ll first need 3 things, your S3 access key, secret key and the bucket you want to put the content in.  I store these things in a database table called SystemSetting.

Next create a set of AWSCredentials from your access and secret keys, then instantiate  a new S3Service.

I then check for the existence of my required bucket, and if the bucket does not exist the create it.

Next comes the application code, it validates the contentType supplied is a valid type (I only support jpg/png/gif).

Then the app generates a random filename and does a check on the s3 service to see if that name already exists.

Then we’re ready to store the file.  Create a new S3Object, set its content and store it using putObject on the S3Service to post it to Amazon.

My app then stores references to the media item in the Media table.

That’s it.  I now have a URL that contains a public link to the image uploaded.

Configuring Cloud Front

The only thing that I haven’t talked about here is the configuration of the CloudFront service.  The code so far takes a byte[] and uploads it to an S3 bucket.  By setting the ACL to REST_CANNED_PUBLIC_READ that makes the file public, and so would be directly accessible from an s3 type URL anyway.

My bucket is called ‘content.fixturetime.com’ so take the example file I have uploaded 04122008130-sm.jpg it is publically available on http://content.fixturetime.com.s3.amazonaws.com/04122008130-sm.jpg. But that URL is ugly, and not using the CDN that is CloudFront.

The last step was to use the Firefox plug-in S3 Organizer to configure a ‘Distribution’.  All you need to do is setup a distribution on the bucket you need and Amazon will allocate a you a dynamic hostname.  I then just created a new DNS record for content.fixturetime.com and setup a CNAME pointer to the allocated Amazon domain, and we’re done.

The same image linked from the above image is now available on http://content.fixturetime.com/04122008130-sm.jpg. But this time is actually served from regional ‘edge’ locations by Amazon automatically.

I haven’t done any testing on this configuration yet, but it was so easy to setup that I’ll take it just for the clean URL.