Amazon AWS Key Management Service with Android

I recently came across Amazon AWS's new Key Management Service (KMS). This seemed like a pretty cool idea so I thought I'd give it a go and see if I could get Amazon to manage my keys on Android.

The idea is pretty straightforward, Amazon host your secure keys for you, so you can encrypt and decrypt without having to worry about key management and storage. As far as I know KMS currently supports only symmetric encryption.

Here's how to use the Amazon Web Services console to create a Key Management Service (KMS) key:

1. Goto AWS management console and click services, under Security and Compliance click IAM
2. Goto groups on the left menu and click create new group
1. Create a group and name and click next
2. Search for “AWSKeyManagementServicePowerUser” and check it, click next
3. Goto users on the left menu and click add user
1. Give the user a name “kms_user” or something
2. Click programatic access
3. You should see your group in the add user to group section, check this
4. Click next
4. Click Encryption Keys at the bottom
1. Give the key a name
2. Do not give the key any administrators, for this tutorial the account owner will be the only one who can administer this key. Click next.
3. Give your kms_user user account use permissions for the key
4. Click next and finish adding the key
5. Setup Android Studio
   compile 'com.amazonaws:aws-android-sdk-kms:2.2.+'

You should now have an encryption key and be ready to start coding. For the Android part of this I used two Async tasks. You could do this in a service but it needs to be off the main UI thread as it's a network call.

public class AsyncEncrypt extends AsyncTask<String, String, ByteBuffer>{

    private static final String TAG = AsyncEncrypt.class.getSimpleName();

    public interface AsyncEncryptListener {
        void processFinish(ByteBuffer cipherText);
    }

    private AsyncEncryptListener listener;

    @Override
    protected ByteBuffer doInBackground(String... strings) {

        final AWSCredentials creds = new AWSCredentials() {
            @Override
            public String getAWSAccessKeyId() {
                return "xxx";
            }

            @Override
            public String getAWSSecretKey() {
                return "yyy";
            }
        };

        AWSKMSClient kms = new AWSKMSClient(creds);

        String keyId = "zzzzz";
        ByteBuffer bytePlainText = ByteBuffer.wrap(strings[0].getBytes());

        EncryptRequest req = new EncryptRequest().withKeyId(keyId).withPlaintext(bytePlainText);
        ByteBuffer ciphertext = kms.encrypt(req).getCiphertextBlob();

        Log.d(TAG, "onCreate: " + ciphertext.toString());

        return ciphertext;
    }

    @Override
    protected void onPostExecute(ByteBuffer cipherText) {
        super.onPostExecute(cipherText);
        if(listener != null){
            listener.processFinish(cipherText);
        }
    }

    public void setListener(AsyncEncryptListener listener){
        this.listener = listener;
    }
}

Decrypt:

public class AsyncDecrypt extends AsyncTask<ByteBuffer, String, String>{

     private static final String TAG = AsyncDecrypt.class.getSimpleName();

     public interface AsyncDecryptListener {
        void processFinish(String plainText);
    }

     private AsyncDecryptListener listener;

     @Override
    protected String doInBackground(ByteBuffer... ciphertextBlob) {

         final AWSCredentials creds = new AWSCredentials() {
            @Override
            public String getAWSAccessKeyId() {
                return "xxx";
            }

             @Override
            public String getAWSSecretKey() {
                return "yyy";
            }
        };

         AWSKMSClient kms = new AWSKMSClient(creds);

         DecryptRequest req = new DecryptRequest().withCiphertextBlob(ciphertextBlob[0]);
        ByteBuffer plainText = kms.decrypt(req).getPlaintext();

         String decoded = new String(plainText.array());
        Log.d(TAG, "onCreate: " + decoded);

         return decoded;
    }

     @Override
    protected void onPostExecute(String plainText) {
        super.onPostExecute(plainText);
        if(listener != null){
            listener.processFinish(plainText);
        }
    }

     public void setListener(AsyncDecryptListener listener){
        this.listener = listener;
    }
}

That's pretty much it. You create a set of AWS credentials supplying the security data given to you in the console for your user, then pass those credentials to the KMS client and make an encryption request.

The only other thing you might want to consider is whether it's worth it or not, in order to use KMS on Android you've got to store your secret key and access key somewhere. If an attacker can get those, they can access your encryption key. It's the classic chicken and egg scenario that distributed systems suffer from again and again. Oh well, it was a neat experiment.

At least it means it's much easier to rotate keys without having to re-release a new app!

Amazon AWS - Signature Version 4

If you decide to try and interact with AWS Glacier API or certain other AWS services you will need to interact with their signature version 4 authentication. Unfortunately in ColdFusion this is one of the hardest things I've ever had to do. Not really ColdFusion's fault, and not really Amazon's fault. Their documentation is comprehensive (although a little confusing) it is just incredibly fiddly. Hashing is a process where a single wrong character completely changes everything. So one slip up causes failures and it can be difficult to determine what you've done wrong.

I have previously blogged about AWS Signiature Version 2 and using it with AWS SES. This article also touches on HMAC and a few of the other key concepts:
http://webdeveloperpadawan.blogspot.co.uk/2012/02/coldfusion-and-amazon-aws-ses-simple.html


AWS Glacier - http://aws.amazon.com/glacier/

AWS Glacier is a very low cost storage solution designed for archiving and backing up data. The basic idea is its cheaper than S3 storage but access is limited. So you don't necessarily have immediate access to your backups. Instead access can be requested and files retrieved within a given time period.

In order to keep costs low, Amazon Glacier is optimized for data that is infrequently accessed and for which retrieval times of several hours are suitable.
- AWS Website

In order to make an API request to Glacier you are required to authenticate each request using their V4 Signature process. Data in Glacier is stored in "Vaults", similar to an S3 bucket a vault is a storage container. For the purposes of this demo I've created a vault using the AWS web management interface. I will be using the API to list all available vaults. In time I hope to expand tutorials and code to cover more complex operations. However, once you've got the signature sorted that shouldn't be too hard.

Code Glorious Code

Again I just want to make the point that I'm just addressing the signature here. I hope to expand the CFC to better deal with making full requests. That will come in time though.

Setup
This should be fairly self explanatory. 

variables.dteNow                  = DateAdd("s", GetTimeZoneInfo().UTCTotalOffset, now());
variables.strPublicKey            = "publickey";
variables.strPrivateKey           = "secretkey";

variables.oAwsSig4                = createObject("component","lib.awsSig4").init(strSecretKey = variables.strPrivateKey);

//We need a few custom date formats
variables.strCanonicalDate        = variables.oAwsSig4.getCanonicalDateFormat(dteNow = variables.dteNow);
variables.strShortDate            = variables.oAwsSig4.getShortDateFormat(dteNow = variables.dteNow);

Step 1 - Create A Canonical Request

The canonical request is basically a standard way to describe the request you are making to AWS. Be that a GET, POST to Glacier or whatever.


This is the aws documentation pseudocode that describes what's happening:

CanonicalRequest =
  HTTPRequestMethod + '\n' +
  CanonicalURI + '\n' +
  CanonicalQueryString + '\n' +
  CanonicalHeaders + '\n' +
  SignedHeaders + '\n' +
  HexEncode(Hash(Payload))

Although I think that's slightly wrong. In my example1 (below), the second empty line is unexplained.

Here's the call I make to the method:

//step 1, create canonical request
strCanonical    = variables.oAwsSig4.createCanonical(
    strHTTPRequestMethod       = "GET",
    strCanonicalURI            = "/-/vaults",
    strCanonicalQueryString    = "",
    arrCanonicalHeaders        = ["date:#variables.strCanonicalDate#","host:glacier.us-east-1.amazonaws.com","x-amz-glacier-version:2012-06-01"],
    arrSignedHeaders           = ["date","host","x-amz-glacier-version"],
    strPayLoad                 = ""
);

You'll note the payload is empty, so the hash at the bottom is simply a SHA-256 hash of "".

The finished Canonical request should look like this.
Example1:

GET
/-/vaults

date:2013-06-26T13:07:03
host:glacier.us-east-1.amazonaws.com
x-amz-glacier-version:2012-06-01

date;host;x-amz-glacier-version
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Step 2 - Create A String To Sign

The string to sign is a little harder to explain out of context. It's basically the message we will hash which will authorize the request to AWS. It's a short and simple string with a very concise format.

The pseudocode is fine here and describes it quite well:

StringToSign  =
Algorithm + '\n' +
RequestDate + '\n' +
CredentialScope + '\n' +
HexEncode(Hash(CanonicalRequest))

Here's my function call:

//Step 2 - Create String To Sign    
strStringToSign    = variables.oAwsSig4.createStringToSign(
    strAlgorithm        = "AWS4-HMAC-SHA256",
    strRequestDate        = variables.oAwsSig4.getStringToSignDateFormat(dteNow = variables.dteNow),
    strCredentialScope    = strShortDate & "/us-east-1/glacier/aws4_request",
    strCanonicalRequest    = strCanonical
);

This is the finished string to sign:

AWS4-HMAC-SHA256
20130626T133038Z
20130626/us-east-1/glacier/aws4_request
6d26f46dbf5d48665e06f44a2f9a65368b3b8d9ef45638b1496fbbe6604ed9db

Step 3a - Calculate Signing Key

OK Now it gets complicated. This is where we start running things through the HMAC function and problems quickly occur. If you do it right though, it'll all come together. AWS do this all in one step, but I think its easier as two.

Here's the documentation pseudocode:

kSecret = Your AWS Secret Access Key
kDate = HMAC("AWS4" + kSecret, Date)
kRegion = HMAC(kDate, Region)
kService = HMAC(kRegion, Service)
kSigning = HMAC(kService, "aws4_request")

Here's my function call:

//create singing key
bSigningKey    = variables.oAwsSig4.createSigningKey(
    dateStamp    = strShortDate,
    regionName    = "us-east-1",
    serviceName    = "glacier"
);

and here's the function:

<cffunction name="createSigningKey" access="public" returnType="binary" output="false" hint="THIS WORKS DO NOT FUCK WITH IT.">
    <cfargument name="dateStamp"    type="string"    required="true" />
    <cfargument name="regionName"    type="string"    required="true" />
    <cfargument name="serviceName"    type="string"    required="true" />
    <cfscript>
        var kSecret     = JavaCast("string","AWS4" & variables.strSecretKey).getBytes("UTF8");
        var kDate       = HMAC_SHA256_bin(arguments.dateStamp, kSecret);
        var kRegion     = HMAC_SHA256_bin(arguments.regionName, kDate);
        var kService    = HMAC_SHA256_bin(arguments.serviceName, kRegion);
        var kSigning    = HMAC_SHA256_bin("aws4_request", kService);
        
        return kSigning;
    </cfscript>
</cffunction>

I know I've not included the function in the other steps, but I want to highlight the importance of two things:

• kSecret is "AWS4" + Secret Key, then cast into bytes. This is a very important step and where I was going wrong for quite a while.
• variables.strSecretKey is the secret key you get from AWS in your account section. It's obviously secret and shouldn't be disclosed to anyone. In my example it's set in the variables scope of the cfc.
• The function I use HMAC_SHA256_bin accepts a binary argument as param1. This is different from the example in my previous blog post on signature version 2, which used two strings as arguments.

So you can see with four HMAC steps, the tiniest mistake means a totally different response. Obviously AWS will be doing the same thing on their end, so if the two don't match, your request won't get approved.

Step 3b - Sign it!

OK Now we bring it all together

signature = HexEncode(HMAC(derived-signing-key, string-to-sign))

We take the signing key from step 3a and the string to sign from step 2:

bSignature    = variables.oAwsSig4.HMAC_SHA256_bin(strStringToSign, bSigningKey);

Step 4 - Put it all together

Now we make our request:

<cfhttp method="GET" url="http://glacier.us-east-1.amazonaws.com/-/vaults">
    <cfhttpparam type="header"         name="Date" value="#variables.strCanonicalDate#">
    <cfhttpparam type="header"        name="x-amz-glacier-version" value="2012-06-01" />
    <cfhttpparam type="header"        name="Authorization" value="AWS4-HMAC-SHA256 Credential=#variables.strPublicKey#/#variables.strShortDate#/us-east-1/glacier/aws4_request,SignedHeaders=date;host;x-amz-glacier-version,Signature=#lcase(binaryEncode(bSignature, 'hex'))#" />
</cfhttp>

• Note we didn't hex encode our bSigniature from before, so I do that in the value. Probably better done elsewhere, but I'll get to it.
• Also note the public key. Again this comes from AWS management console.
• Note the three different dates, one is the strCanonicalDate we created at the top, the other is the short date and finally the hard coded glacier version date.

That's it. That should work!  Obviously you need the cfc. Take a look below for that.


Advice

My advice to anyone attempting to do this in ColdFusion or any other language is as such:

1. Baby Steps - The documentation is presented in steps. Get each step working perfectly before moving onto the next. They all rely on each other, so a mistake early on will just cascade and waste time later.
2. Unit Tests - I'm a huge fan of unit tests anyway, but in this case they really helped. Setting up some great unit tests using AWS examples will help you define your input and your output and tweak your code until you get the response you're looking for.
3. Check Responses - If you actually make a request to AWS they will tell you in the response what the problem is. Plus they'll tell you the expected canonical request and the expected string to sign. These can really help iron out any tiny discrepancies
4. Try it in Java -  CFML Doesn't have a native HMAC function (pre CF10), and converting too and from byte arrays caused endless problems. So I just did a few piecemeal functions in Java and learned from that.

The CFC

Ah of course one final step. The cfc. As I've said I hope to improve it a great deal and perhaps open source it and put it on cflib. I've also got a bunch of unit tests I wrote which may help people improve it.
Right now I'm just pleased to have got it working and don't want to forget it all so it's going on the blog.


Thanks

I ended up not using the code, but some of Ben Nadel's stuff was really useful to understand. He's written a great cfc that I urge anyone using HMAC in CFML to consider:
http://www.bennadel.com/blog/2412-Crypto-cfc-For-Hmac-SHA1-Hmac-Sha256-and-Hmac-MD5-Code-Generation-In-ColdFusion.htm

<cfcomponent output="false">

<!---
<OWNER> = James Solo
<YEAR> = 2013

In the original BSD license, both occurrences of the phrase "COPYRIGHT HOLDERS AND CONTRIBUTORS" in the disclaimer read "REGENTS AND CONTRIBUTORS".

Here is the license template:

Copyright (c) 2013, James Solo
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--->

    <cffunction name="init" access="public" returnType="awsSig4" output="false">
        <cfargument name="strSecretKey" type="string" required="true" />

        <cfset variables.strSecretKey        = arguments.strSecretKey />
        <cfset variables.strPublicKey        = "mypublickey" />
        <cfreturn this>
    </cffunction>


    <cffunction name="createCanonical" access="public" returnType="string" output="false" hint="Create the canonical request">
        <cfargument name="strHTTPRequestMethod"        type="string"    required="true"                 />
        <cfargument name="strCanonicalURI"            type="string"    required="true"                 />
        <cfargument name="strCanonicalQueryString"    type="string"    required="false"    default=""    />
        <cfargument name="arrCanonicalHeaders"         type="array"    required="true"                    />
        <cfargument name="arrSignedHeaders"            type="array"    required="true"                    />
        <cfargument name="strPayload"                type="string"    required="false"    default=""    />
        
        <cfscript>
            var intCount            = 0;
            var strHeaderString        = "";
            var strCanonicalRequest = 
                arguments.strHTTPRequestMethod        & chr(010) &
                arguments.strCanonicalURI            & chr(010) &
                arguments.strCanonicalQueryString    & chr(010);
            
            //Headers
            for(intCount=1; intCount <= arraylen(arrCanonicalHeaders); intCount++){
                strCanonicalRequest    &= arguments.arrCanonicalHeaders[intCount] & chr(010);
            }
            
            strCanonicalRequest    &= chr(010);
            
            //Signed headers
            for(intCount=1; intCount <= arraylen(arrSignedHeaders); intCount++){
                strHeaderString        = arguments.arrSignedHeaders[intCount];
                strCanonicalRequest    &= strHeaderString;
                
                //put a semi-colon between headers, or a new line at end
                if(intCount EQ arraylen(arrSignedHeaders)){
                    strCanonicalRequest    &= chr(010);
                }else{
                    strCanonicalRequest    &= ";";
                }
            }
            
            strCanonicalRequest    &= lcase(hash(arguments.strPayload, "SHA-256"));
            
            return trim(strCanonicalRequest);
        </cfscript>
    </cffunction>


    <cffunction name="createStringToSign" access="public" returnType="string" output="false" hint="I create the string to sign">
        <cfargument name="strAlgorithm"            type="string" required="true" />
        <cfargument name="strRequestDate"        type="string" required="true" />
        <cfargument name="strCredentialScope"    type="string" required="true" />
        <cfargument name="strCanonicalRequest"    type="string" required="true" />
        
        <cfscript>
            var strStringToSign  =
                    arguments.strAlgorithm            & chr(010) &
                    arguments.strRequestDate        & chr(010) &
                    arguments.strCredentialScope    & chr(010) &
                    lcase(hash(arguments.strCanonicalRequest, "SHA-256"));
            
            return strStringToSign;
        </cfscript>
    </cffunction>


    <cffunction name="createSigningKey" access="public" returnType="binary" output="false" hint="THIS WORKS DO NOT FUCK WITH IT.">
        <cfargument name="dateStamp"    type="string"    required="true" />
        <cfargument name="regionName"    type="string"    required="true" />
        <cfargument name="serviceName"    type="string"    required="true" />
        <cfscript>
            var kSecret        = JavaCast("string","AWS4" & variables.strSecretKey).getBytes("UTF8");
            var kDate        = HMAC_SHA256_bin(arguments.dateStamp, kSecret);
            var kRegion        = HMAC_SHA256_bin(arguments.regionName, kDate);
            var kService    = HMAC_SHA256_bin(arguments.serviceName, kRegion);
            var kSigning    = HMAC_SHA256_bin("aws4_request", kService);
            
            return kSigning;
        </cfscript>
    </cffunction>


    <cffunction name="HMAC_SHA256_bin" access="public" returntype="binary" output="false" hint="THIS WORKS DO NOT FUCK WITH IT."> 
        <cfargument name="signMessage"    type="string" required="true" />
        <cfargument name="signKey"        type="binary" required="true" /> 
        
        <cfset var jMsg = JavaCast("string",arguments.signMessage).getBytes("UTF8") /> 
        <cfset var jKey = arguments.signKey />
        
        <cfset var key = createObject("java","javax.crypto.spec.SecretKeySpec") /> 
        <cfset var mac = createObject("java","javax.crypto.Mac") /> 
        
        <cfset key = key.init(jKey,"HmacSHA256") /> 
        
        <cfset mac = mac.getInstance(key.getAlgorithm()) /> 
        <cfset mac.init(key) /> 
        <cfset mac.update(jMsg) /> 
        
        <cfreturn mac.doFinal() />
    </cffunction>


    <cffunction name="toHex" access="public" returnType="string" output="false" hint="I convert binary to hex">
        <cfargument name="bSignature"        type="binary" required="true" /> 
        <cfreturn lcase(binaryEncode(arguments.bSignature, "hex")) />
    </cffunction>
    

    <cffunction name="getCanonicalDateFormat" access="public" returnType="string" output="false" hint="I return a formatted date time for the canonical part of the process">
        <cfargument name="dteNow"    type="date"    required="true" />
        
        <cfreturn "#dateformat(arguments.dteNow, 'yyyy-mm-dd')#T#TimeFormat(arguments.dteNow, 'HH:mm:ss')#" />
    </cffunction>
    

    <cffunction name="getStringToSignDateFormat" access="public" returnType="string" output="false" hint="I return a formatted date time for the string to sign section">
        <cfargument name="dteNow"    type="date"    required="true" />
        
        <cfreturn "#dateformat(arguments.dteNow, 'yyyymmdd')#T#TimeFormat(arguments.dteNow, 'HHmmss')#Z" />
    </cffunction>
    

    <cffunction name="getShortDateFormat" access="public" returnType="string" output="false" hint="I return a short date time">
        <cfargument name="dteNow"    type="date"    required="true" />
        
        <cfreturn "#dateformat(arguments.dteNow, 'yyyymmdd')#" />
    </cffunction>


</cfcomponent>

Install and setup of mysql on an amazon EC2 Instance.

Amazon RDS costs me quite a bit and I utilize about 1% of its true potential. That's not to say it isn't a great service (it totally is), I just figured I could do it cheaper by running mysql on the same instance as my web server. Might be cheaper, might not, I'll see. It certainly won't be as straightforward as RDS which is super simple.

--setup db
sudo yum install mysql
sudo yum install mysql-server
sudo yum install mysql-devel
sudo chgrp -R mysql /var/lib/mysql
sudo chmod -R 770 /var/lib/mysql
sudo service mysqld start
/usr/bin/mysqladmin -u root password yourpasswordhere

Please note, my password is actually not "yourpasswordhere". Neither should yours be ;)
Now we must login to mysql and create the db

mysql -u root -p

mysql> CREATE DATABASE xxxx;
mysql> exit

Again, don't call your db xxxx, unless it's some very hardcore stuff!
Now if you're doing this with Railo you can setup your datasource in railo server admin.
http://xx.xx.xx.xxx/railo-context/admin/server.cfm

That's it, you're all ready to go.

Hats off Sam Starling from whom I figured this out:
http://www.samstarling.co.uk/2010/10/installing-mysql-on-an-ec2-micro-instance/
I've said it before and I'll say it again, if only there was a decent GUI for connecting to mysql and <cough> ms sql. <sigh>

Connect to EC2 Instance from unix

I've spent a few days slightly mystified by not being able to connect to my Amazon AWS EC2 instance from my Ubuntu machine. Sadly I've fixed it now, so I can't re-type the error. Slightly concerning you may scream, but I could connect from a windows machine and I don't have any visitors anyway...so meh!

Anyway after much researching I stumbled across this:

chmod 400 mykeyname.pem 

This worked a charm, I can now connect via ssh or the java browser util that Amazon provide. Awesome! If this is a new AWS security change or not I don't know. Anyway, hope this helps someone!

Connect Amazon EC2 Instance to RDS DB

This cost me some time and by the looks of some Google searches it cost a few other people time too.

You need to first click on DB Security Groups and add the Elastic IP of your EC2 instance as an CIDR. The important bit that I missed is you also need to add the EC2 Security Group that your EC2 instance is configured with.

This guy figured it out:
http://chris-allen-lane.com/2011/07/amazon-ec2-instance-cannot-connect-to-amazon-rds-database-server/

Dropbox integration with EC2 Linux instance

So Dropbox has a great linux command line tool which comes in very useful for copying files to your Amazon linux EC2 instance, without having to open up FTP or copying each file by hand. It took me ages figuring out how to set this up, but it saves a lot of time.

First we download dropbox and unzip it:

$ wget -O dropbox.tar.gz "http://www.dropbox.com/download/?plat=lnx.x86"
$ tar -xvzf dropbox.tar.gz

Run it:

$ ~/.dropbox-dist/dropboxd &

..and you should see a message like this:
"This client is not linked to any account... Please visit https://www.dropbox.com/cli_link?host_id=XXXXX to link this machine."
Copy/paste that URL into a Web browser on your local machine; log into dropbox; and voila! The directory ~/Dropbox will be linked into your home directory! The repeating message on your console should then stop. If it doesn't press Ctrl+C.

You should now see a new folder called "Dropbox" sync'd with your dropbox account.

Now we need a package called dropbox.py, this is the dropbox command line tool. Very useful, if a little tricky to use sometimes.

$ wget -O ~/dropbox.py "http://www.dropbox.com/download?dl=packages/dropbox.py"

First there's a few folders we don't want sync'd so we add them to the exclude list.

$ python dropbox.py exclude add ~/Dropbox/Public   
Excluded: 
Dropbox/public
$ python dropbox.py exclude add ~/Dropbox/Photos 
Excluded: 
Dropbox/photos

Now we're going to add a symbolic link. This links our dropbox folder to our web root.

$ python dropbox.py dropbox stop                   
Dropbox daemon stopped.
$ sudo mv ~/Dropbox /opt/railo/tomcat/webapps
$ ln -s /opt/railo/tomcat/webapps/Dropbox ~/
$ python dropbox.py dropbox start
Starting Dropbox...Done!

Setup an AWS EC2 Instance Running Railo

Setup an AWS EC2 Instance Running Railo

OK so you want to delve deep into cloud computing and start your own instance Amazon Web Services (AWS) Elastic Cloud Compute (EC2) instance? Sadly the “official” Railo AMI seems to have died, so here we’ll be starting our own new one. We’ll be running firmly within the free criteria here and choosing options appropriately, most significantly this means linux! We’ll also be using putty to connect to our instance. For part of this tutorial we’re going to be running alongside the official Amazon starting an instance guide. So I will skimp on the details already covered by Amazon themselves:

http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/GetStartedLinux.html

Start-up an Instance

1. Click the giant “Launch an Instance” button. Select Basic Amazon Linux AMI I choose 32bit because it’s cheaper and (at the moment) free.
2. Ensure you’ve selected a micro instance.
3. Skip the instance details section, just accepting the defaults.
4. Create and download a Key Pair. This is important as it allows us to log onto our instance securely.
5. Next is the firewall or security groups section. This bit is important as it configures what applications and ports are allowed to access your instance. Create a new security group. You should select SSH and HTTP as a minimum, you can accept the default of 0.0.0.0 but that allows any IP access to these ports. This is fine for HTTP but if your ISP has given you a static IP then put this in for SSH.
6. Done, your instance will begin powering up. Watch the instances dashboard to see it’s status, eventually the status will flick to green, display “running” and the status checks will show 2/2. Once that happens we’re ready to logon.

Connect to your instance
I’m going to leave this bit a little to Amazon to explain. You’ll need to download and install putty and convert your key pair file (from step 4 above) into a putty private key file. Then grab your amazon public dns value (something like ec2-11-11-111-111.compute-1.amazonaws.com) and connect to it using putty. Don’t forget to enter the username ec2-user.

Get Linux straight & Install Railo

1. First lets get Linux to update itself:

sudo yum update

2. Download Railo to the instance:

wget http://www.getrailo.org/down.cfm?item=/railo/remote/download/3.3.1.000/tomcat/linux/railo-3.3.1.000-pl1-linux-installer.run

3. Assign permissions

sudo chmod 777 railo-3.3.1.000-pl1-linux-installer.run

4. Run Railo

sudo ./railo-3.3.1.000-pl1-linux-installer.run

Here you’ll want to accept all the defaults except three. First change the default password to something good. Second you should set the port to 80 (not 8888). This will allow normal connections to your server and links into why we had to allow HTTP (port 80) in the EC2 security group. Lastly you should say no to the apache connectors. This sets up railo with tomcat and installs tomcat for you. Of course if you’re more familiar with apache then go with that.

Hit your url
http://ec2-11-11-111-111.compute-1.amazonaws.com

The above link (customized for your public DNS) should show you the default Railo welcome page.

Use putty and VI to change your cfm files

1. Navigate to the webroot:

cd /opt/railo/tomcat/webapps/ROOT/

2. Remove all these files

sudo rm -rf *

3. Create a new index.cfm file

sudo vi index.cfm

VI - Linux editor

1. In vi to delete the contents of the whole file type

:1,$d

2. To swap between command and insert modes just press escape


3. To exit without saving, switch to command mode and press

:q!

4. To exit and save switch to command mode and press

:x

ColdFusion and Amazon AWS SES - Simple Email Service

Amazon Web Services (AWS) have a service called Simple Email Service (SES), SES is an excellent e-mail sending service for businesses and developers. It enables developers to quickly and easily send emails without having to go through the pains of setting up an SMTP server or any of that nonsense. It is intended for bulk sending marketing material, but you can also use it in very low volumes and it becomes a great tool.

To demonstrate the service, we’re going to use SES to send an email to ourselves using coldFusion. First and foremost you’ll need to sign up to SES (https://aws.amazon.com/ses/) and verify an email address. SES starts up in “development mode” where you have to verify every email address you want to send mail too. This is a fantastic feature that prevents accidental sending of email and lets you test out the functionality. For the purpose of this example development mode is perfect, so sign up and verify your own email address.

There are quite a few ways to get SES to send an email for you, we’re going to use what I think is the easiest to get started with, a POST request. It’s basically just liked submitting a form, but we’re going to use cfhttp to mimic a form post. First though, there’s a few things we need to setup before we actually do any coding.

1) Security details
Obviously it would be a pretty crummy service if anyone could use your email service to send emails to anyone, spam would run rampant! So there a few security details we need to get from amazon to authenticate each SES request. I won’t go into too much detail but you’ll need an “Access Key ID” and a “Secret Access Key”. You get these from amazon by logging into your aws account and clicking security credentials from the top right drop down. Look at this page for a bit more info:
http://docs.amazonwebservices.com/ses/latest/DeveloperGuide/QueryInterface.Authentication.html#QueryInterface.Authentication.Signatures

2) HMAC - Hash-based Message Authentication Code
http://en.wikipedia.org/wiki/HMAC
To authenticate our message Amazon requires use of a security algorithm called HMAC. The theory being if we (us and Amazon) have a shared private key, we can encrypt a known variable and authenticate ourselves. This is a very very brief introduction if you’ve never come across HMAC. HMAC is a cryptography method of creating a hash using a secret key. Normally a hash takes any value and creates a fixed length string. For example, using the MD5 hash in CF:

 hash(‘all your base are belong to us’)  

would return
847DBEB849668D30722D8A67BCED1C59

With HMAC we introduce a secret key to add an extended element of cryptography and security to the proceedings. Here’s an example:
 

 toString(toBase64(HMAC_SHA1('password','all your base are belong to us')))  

z0A498Rx0aNzXNOwfbjSKkhBdbA=

HMAC can use MD5 or SHA1 and Amazon does support both, but we’re going to use SHA1. Now coldFusion doesn’t offer the HMAC function by default, but that’s not to say we can’t write or borrow our own..

 <cffunction name="HMAC_SHA1" returntype="binary" access="public" output="false" hint="I create an HMAC hashed string from a given message and secret key.">  
      <cfargument name="signKey" type="string" required="true" hint="Secret key with which to encrypt message" />  
      <cfargument name="signMessage" type="string" required="true" hint="Message you want to hash"/>  
      <cfset var jMsg = JavaCast("string",arguments.signMessage).getBytes("iso-8859-1") />  
      <cfset var jKey = JavaCast("string",arguments.signKey).getBytes("iso-8859-1") />  
      <cfset var key = createObject("java","javax.crypto.spec.SecretKeySpec") />  
      <cfset var mac = createObject("java","javax.crypto.Mac") />  
      <cfset key = key.init(jKey,"HmacSHA1") />  
      <cfset mac = mac.getInstance(key.getAlgorithm()) />  
      <cfset mac.init(key) />  
      <cfset mac.update(jMsg) />  
      <cfreturn mac.doFinal() />  
 </cffunction>  

3) Create your Amazon authenticated header

Next we want to create our HMAC hash to pass to Amazon, they require us to encrypt the date and time, so to do so looks a little like this:

 <cfset variables.dteOldDate = now() />  
 <cfset variables.strDateString = "#DateFormat(variables.dteOldDate, 'ddd, dd mmm yyyy')# #timeFormat(variables.dteOldDate,'HH:mm:ss')# GMT" />  
 <!--- encrypt date --->  
 <cfset variables.xxxHMac = toString(toBase64(HMAC_SHA1(“yoursecretaccesskeygoeshere”,variables.strDateString))) />  

Next we need to integrate our hashed date with a header called a X-Amzn-Authorization

 <cfset variables.strAmazonHeader = "AWS3-HTTPS AWSAccessKeyId=#youraccesskeyidgoeshere#, Algorithm=HmacSHA1, Signature=#xxxHMac#" />  

So obviously replace youraccesskeyidgoeshere with the Access Key you got from Amazon.


4) Make the httpd call

That’s the complicated bit done, now all we have to do is make the cfhttpd call.

 <cfhttp url="https://email.us-east-1.amazonaws.com/" method="post" result="result">  
      <cfhttpparam type="header"       name="X-Amzn-Authorization" value="#variables.strAmazonHeader#" />  
      <cfhttpparam type="header"       name="Date" value="#variables.strDateString#" />  
      <cfhttpparam type="formfield"      name="Action" value="SendEmail" />  
      <cfhttpparam type="formfield"      name="Destination.ToAddresses.member.1" value="#yourToEmailAddressGoesHere#" />  
      <cfhttpparam type="formfield"      name="Message.Body.Text.Data" value="#yourEmailContentGoesHere#" />  
      <cfhttpparam type="formfield"      name="Message.Subject.Data" value="#yourEmailSubjectGoesHere#" />  
      <cfhttpparam type="formfield"      name="Source" value="yourFromEmailGoesHere" />  
 </cfhttp>   

That's it, run your code and you should receive an email!

Assigning your GoDaddy domain name to your Amazon S3 bucket

In the previous post we setup a static website using Amazon Web Services (AWS) S3 bucket storage. Now we’re going to assign the fancy new godaddy domain we just bought to that bucket.
You will need:
    - A GoDaddy Domain
    - An working S3 bucket hosting your website

1. Bucket Name
First and foremost in the last post we named our bucket whatever we want. In this post it’s crucially important we are very careful with what we name our bucket. Amazon gets the bucket name to use from the URL, so you’ll need to match your domain with your bucket name. For example if my domain was www.mygreatwebsite.com you would need to make sure your bucket was called www.mygreatwebsite.com So if you haven’t already done that, go ahead and create a new bucket. You can easily copy and paste your files across from your old bucket. Of course make sure it’s setup as a website all works with the S3 URL.

2. GoDaddy Domain Manager
First we need to open what GoDaddy call the zone manager, there may be better ways to find it but here’s how i get there. Open godaddy.com, hover the mouse over Domains and click “Domain Manager” and login. Hover over Tools and click “DNS Manager”. In the list of your domains click “Edit Zone” next to your domain. You’ll see a bunch of information here, but in short you’re looking under the CNAME heading for www. By default it’s value is @. You need to remove this at symbol and change it to: mygreatwebsite.com.s3-website-eu-west-1.amazonaws.com
Obviously replacing the url with the one for your S3 bucket. Especially the “eu-west” bit and the mygreatwebsite.com with yourgreatwebsite.com and save. This is where the name of your bucket we mentioned in step one becomes so important.

You may want to check you don’t have forwarding turned on. The first time I set this up I just used domain forwarding, which is a neat and easy to use feature, but it just creates an iframe for your site, which won’t help your SEO one little bit. That said, it is a very quick and simple to use feature of GoDaddys. You can turn domain forwarding on and off in the DSN manager using the forward button at the top.

If you own http://mysite.com and http://www.mysite.com then (I believe) you have to either setup two buckets one called mysite.com and one called www.mysite.com or of course you could forward one domain to the other.

That’s it, sit back and be patient, domain changes can take a few hours to process so go buy some donuts and cross your fingers. Hopefully that all worked like a dream.

I owe a lot to this guy, Aaron Blohowiak and his excellent blog entry: http://aaronblohowiak.com/using-amazon-s3-and-cloudfront-to-host-a-stat

and also this this one by Steve Liles:
http://steveliles.github.com/pointing_a_domain_name_to_an_amazon_s3_bucket.html