Photo backup and sync using Google Cloud Bucket Storage

This morning I spotted a tweet by Greg Wilson whom I don't know and have never met. However it appears he's some kind of director at Google Cloud, so that might explain why I'm following him and why he tweeted this:

https://twitter.com/gregsramblings/status/1284743960955510787?s=20

Archiving my newly organized 238k+ photo library (2.2TB) to Google Cloud Storage with: gsutil -m rsync -r -d . gs://{mybucketname} I'm using crazy-cheap 'archive' storage class in single regionStorage price: $0.0012/GB/Month (!) @gcpcloud

This really interested me as I've been trying to upload photos to an AWS (Amazon Web Services) bucket recently but it's been a tedious process.

I am aware that various cloud services like dropbox and pcloud exist, but I want the following:

1. Encryption
2. Ease of use
3. Cheap

I really only want this as a backup in case of fire or failure of one of my crummy old USB HDDs fails. I genuinely don't understand why so many of these services make such a bad job of encryption or offer it as some kind of bizarre add on. Come on people this is 2020, it's not hard and it shouldn't be expensive.

So I realised with some basic AWS skills I can spin up a bucket running glacier for pennies a month. This worked really really well, but the upload tools are slow and not terribly reliable for bulk uploads. Which I guess is understandable because the whole point of AWS is to build your own right?

Back to my original point, I stumbled across this tweet and thought hmm, that sounds nice and easy. Let's give it a go on a dull lockdown Sunday.

The following I followed from here: https://cloud.google.com/storage/docs/quickstart-gsutil

1. Install Python 3.8

Did someone say snakes on a plane?

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.8

2. Download & Extract Google Cloud SDK
https://cloud.google.com/sdk/docs

3. Create a bucket
Then I created a new Google Cloud Project on their cloud console and a new Cloud Storage Bucket. This was actually really easy and the bucket creation process easily walked me through the access policies and encryption setup. Although to be fair the AWS process is pretty good now as well. Don't forget to select your preferred storage type but bare in mind different types have different latency, pricing and minimum storage commitments.
https://cloud.google.com/storage/pricing#operations-pricing

4. Initialise the sdk
Run gcloud init this will initialise your sdk and link it to your google account. You can then of course select the project you're working on.

5. Copy a file
That's pretty much it, from there you can copy individual files using gloud cp or as Greg's tweet suggest you can use the sync option for bulk upload.

This is such a simple process I was really impressed.

One thing it's important to note is that Google Storage buckets may have a minimum storage time period:
https://cloud.google.com/storage/pricing#archival-pricing

Firebase Realtime Database - Custom Rules - Admin Write Permissions

Firebase is cool. Firebase realtime database is also cool. Rules...they are way less cool.

I joke, but protecting your Firebase realtime data is of course important and not to be under-estimated. I wanted to setup Firebase realtime database rules so only the database administrator (me) could enter or edit data. This took me a while to figure out as it involves not only setting the correct rules, but editing the metadata of the user.

Right so let's jump straight in, we're going to set the database rules to allow only edits by an administrator:

"rules":{
        "dinosaurs":{
            ".read":"auth != null",
            ".write":"auth.token.admin === true"
        }

You can see we've granted write permissions only if the user is an administrator. That was pretty easy. Sadly actually configuring which user is an admin is much harder.

First goto your Firebase console and click on the Authentication section, you should be on the user's tab. Locate the user you want to make an administrator and copy their User UUID.

To set this user as an administrator you'll need to use the Firebase Admin SDK. This isn't an Android SDK and so I'm afraid we're going to have to break into some NodeJS.

I'm not going to go into setting up NodeJS here, as it's a huge pain in the neck. Once you've got NodeJS setup, install the firebase Admin SDK as such:
https://firebase.google.com/docs/admin/setup?authuser=0

$ npm install firebase-admin --save

Now you'll need to return to your Firebase console and create a service key. In the settings section, goto service accounts and download a private key. You'll need to create one for NodeJS and you'll need to copy it to the NodeJS folder you're about to create. The path from the index.js file needs to be relative, mine is justin the root. As we won't be sharing this project, it should be fine, but you should never publicly host or share your private key.
Now you should be ready to run a NodeJS script which will set this user as an admin.

Create a standard node project and edit the index.js file. Add the following code:

var admin = require('firebase-admin');

var serviceAccount = require('./service_key.json');

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL: '*****'
});


admin.auth().setCustomUserClaims('#######', {admin: true}).then(() => {
// The new custom claims will propagate to the user's ID token the
// next time a new one is issued.
});

Replace the *** with your database url and #### with the UUID you copied earlier and run your nodeJs project.

That's it. Your user should now be an administrator, it can take a while to propagate, so try logging on and off a few times if it doesn't work right away.

What you've done here is set a custom Claim on that user. Basically some meta-data that describes that user as an admin. Your database rules will only allow administrators to edit that data. Here's a bit more info should you wish to take Calims further:
https://firebase.google.com/docs/auth/admin/custom-claims?authuser=0

Good luck, and have fun with Firebase.

Google App Engine and Android playing nice.

I'm a big fan of cloud services, especially when they give you a free or basic quota such as Google App Engine which you can use for development or testing. I’ve done quite a bit of work with Google App Engine before but not in a while. It became a real heavy beast involving importing add ons to Eclipse and configuring a multitude of environment settings.

However recently I've moved to Android Studio and apparently Google Cloud support is built in. Learning this I was then inspired by a recent post on the Android Developers Blog:
http://android-developers.blogspot.co.uk/2014/12/build-mobile-app-services-with-google.html

I felt this wasn't a great tutorial, there were large sections left out and it basically didn't work nearly as easily as I’d hoped it would. Sorry Android Developers, but not your best work. Still the steps to get started were simple enough:

1. Create a super simple Android app. Basic Hello world stuff.
2. Give your app Internet permissions
3. Create a new Google Cloud Module
   • File -> New Module
   • Click “Google Cloud Module”
   • Select App Engine Java Endpoints Module

Android studio will now create a GAE backend module for you and automatically tie it into your Android app. Now you need a bean and an endpoint.

MyBean.java

package com.example.myapplication.backend;

public class MyBean {

    private String myData;

    public String getData() {
        return myData;
    }

    public void setData(String data) {
        myData = data;
    }
}

MyEndpoint.java

package com.example.myapplication.backend;

import com.google.api.server.spi.config.Api;
import com.google.api.server.spi.config.ApiMethod;
import com.google.api.server.spi.config.ApiNamespace;

import javax.inject.Named;

/**
 * An endpoint class we are exposing
 */
@Api(name = "myApi", version = "v1", namespace = @ApiNamespace(ownerDomain = "backend.myapplication.example.com", ownerName = "backend.myapplication.example.com", packagePath = ""))
public class MyEndpoint {

    /**
     * A simple endpoint method that takes a name and says Hi back
     */
    @ApiMethod(name = "sayHi")
    public MyBean sayHi(@Named("name") String name) {
        MyBean response = new MyBean();
        response.setData("Hi, " + name);

        return response;
    }

}

and here’s the Gradle file:

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'com.google.appengine:gradle-appengine-plugin:1.9.14'
    }
}

repositories {
    mavenCentral();
}

apply plugin: 'java'
apply plugin: 'war'
apply plugin: 'appengine'

sourceCompatibility = JavaVersion.VERSION_1_7
targetCompatibility = JavaVersion.VERSION_1_7

dependencies {
    appengineSdk 'com.google.appengine:appengine-java-sdk:1.9.14'
    compile 'com.google.appengine:appengine-endpoints:1.9.14'
    compile 'com.google.appengine:appengine-endpoints-deps:1.9.14'
    compile 'javax.servlet:servlet-api:2.5'
}

appengine {
    downloadSdk = true
    appcfg {
        oauth2 = true
    }
    endpoints {
        getClientLibsOnBuild = true
        getDiscoveryDocsOnBuild = true
    }
}

Frustratingly you’ll get red errors all over the place. Cannot resolve symbol api. This really winds me up and I’ve not yet figured out how to fix it. However it does build and run with these problems, so not really an error!

If you change the run drop down to “backend” and hit run hopefully this will all compile and you’ll get an Android Studio message with a localhost url. This is Android Studio setting up a local version of Google App Engine and using Jetty to host it. You should see a url output which you can copy to your browser, something like:
http://localhost:8080/

Hitting this url you should get an index file saying Hello Endpoints or something with a nice pretty bootstrap wrapper. You can enter something into the text field and your GAE application will say Hi to you.

Now we need to plug this into our Android app. First we need a new AsyncTask

package tester.example.com.myapplication;

import android.content.Context;
import android.os.AsyncTask;
import android.support.v4.util.Pair;
import android.widget.Toast;

import com.example.myapplication.backend.myApi.MyApi;

import com.google.api.client.extensions.android.http.AndroidHttp;
import com.google.api.client.extensions.android.json.AndroidJsonFactory;
import com.google.api.client.googleapis.services.AbstractGoogleClientRequest;
import com.google.api.client.googleapis.services.GoogleClientRequestInitializer;


import java.io.IOException;

class EndpointsAsyncTask extends AsyncTask<Pair<Context, String>, Void, String> {
    private static MyApi myApiService = null;
    private Context context;

    @Override
    protected String doInBackground(Pair<Context, String>... params) {
        if(myApiService == null) {  // Only do this once
            MyApi.Builder builder = new MyApi.Builder(AndroidHttp.newCompatibleTransport(),
                    new AndroidJsonFactory(), null)
                    // options for running against local devappserver
                    // - 10.0.2.2 is localhost's IP address in Android emulator
                    // - turn off compression when running against local devappserver
                    .setRootUrl("http://10.0.2.2:8080/_ah/api/")
                    .setGoogleClientRequestInitializer(new GoogleClientRequestInitializer() {
                        @Override
                        public void initialize(AbstractGoogleClientRequest<?> abstractGoogleClientRequest) throws IOException {
                            abstractGoogleClientRequest.setDisableGZipContent(true);
                        }
                    });
            // end options for devappserver

            myApiService = builder.build();
        }

        context = params[0].first;
        String name = params[0].second;

        try {
            return myApiService.sayHi(name).execute().getData();
        } catch (IOException e) {
            return e.getMessage();
        }
    }

    @Override
    protected void onPostExecute(String result) {
        Toast.makeText(context, result, Toast.LENGTH_LONG).show();
    }
}

Now in your app activityMain or somewhere, fire a call to the AsyncTask:

new EndpointsAsyncTask().execute(new Pair(this, "Manfred"));

The dependencies in your Android app should look like this:

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    compile 'com.android.support:appcompat-v7:21.0.3'
    compile 'com.google.android.gms:play-services:6.1.71'
    compile project(path: ':backend', configuration: 'android-endpoints')
}

Now if you run your Android app in a Android Virtual Device, you should see your local server respond with a Hi Message.

This is a pretty basic example, but you get the idea and its a great first step on the path toward Android and Google Cloud playing well together.

Google Cloud Messaging with Android and ColdFusion backend

Google Cloud Messaging (GCM) is an awesome little service. It enabled "push" messages to be sent from Google to a specific Android phone and picked up by your application. The scope for these messages is massive but the intent is for the content to be fairly small. A great example (and a great name) is a tickle. This is very small message intended to instruct the application to go and perform an action.

For example, say your app needs data to be kept in sync with your server. This could be done by polling, but polling is resource intensive and sucks up precious bandwidth. A far better idea would be for your server to send out a message whenever the necessary data is updated and tell your app to come and re-fetch the data when possible. The message sent to your app is called a tickle!

I've created a simple tutorial to briefly describe GCM and show an example. My example does have a lot of moving parts to it but hopefully is simple enough to get the basic idea. For my backend server I'm using a CFML server which keeps track of any device registering for messages and allows me to send a message out too that device.

The sample app uses three components:

1. GCM Connection Server - A Google Cloud project to send the message from you to the device.
2. Application Server - Your own personal server that tracks device ids and issues messages. This can be cfml, php aspx, whatever, but it needs the capacity to store data, preferably in a db.
3. Your Android app - The app you write and distribute as necessary.

Step 1 - The GCM Server

OK First we need to setup our Google Cloud Messaging service. To do this we log onto Google's cloud messaging panel and configure a new project. This project will now sit and wait for instructions and when told will relay messages from our server to our elected android device.

To set this up, follow the instructions here: http://developer.android.com/google/gcm/gs.html

Step 2 - The Application Server

As mentioned for this I'm using CFML here with a mysql db. I've not spent much time on this so it doesn't look pretty but I just wanted to illustrate the point. First the db, I've used one simple table:

CREATE TABLE gcmtest{
 intID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
 strName varchar(100) not null,
 registrationid varchar(200) not null
}

That's it for db work! We have an ID column, a human readable name and a registration id which will contain the device id which is registered with Google.

Now the application server needs two files. One to respond to incoming registration requests and one to send messages, first the registration responder. Now a word of warning, this isn't production worthy, it's just a test bed. You should sanitise and protect your data much more thoroughly. As you can see it takes two values from the URL and inserts them into the db.

receive.cfm:

<cfif structKeyExists(url, "registrationid") AND structKeyExists(url, "name")>

    <cfquery name="qInsertReg" datasource="local">
        insert into gcmtest 
        (
            registrationid,
            strname
        )
        values
        (
            <cfqueryparam cfsqltype="CF_SQL_VARCHAR"    value="#url.registrationid#" />,
            <cfqueryparam cfsqltype="CF_SQL_VARCHAR"    value="#url.name#" />
        )
    </cfquery>

    <cfoutput>Done.</cfoutput>

</cfif>

Next you need the page which will send out messages, send.cfm:

<cfif structKeyExists(url, "regid")>

    <cfif structKeyExists(url, "strmsg")>
        <cfset strmymsg = url.strmsg />
    <cfelse>
        <cfset strmymsg = "all your base are belong to me" />
    </cfif>

    <cfset stFields = { "registration_ids": [ "#url.regid#" ], "data": {msg: "#strmymsg#"} }>
    
    <cfhttp url="https://android.googleapis.com/gcm/send" method="post" result="httpResp" timeout="60">
        <cfhttpparam type="header" name="Content-Type" value="application/json" />
        <cfhttpparam type="header" name="Authorization" value="key=***yourkeyfromgoogle***" />
        <cfhttpparam type="body" value="#serializeJSON(stFields)#">
    </cfhttp>
    
        
    <cfif httpResp.status_code eq 200>
        <span style="color:green;font-weight:bold;">Message Sent</span><br />
    </cfif>
</cfif>


<cfquery name="qGet" datasource="local">
    select * from gcmtest
</cfquery>

<form action="send.cfm" method="GET">
    <strong>Select Recipient:</strong>
    <br />
    
    <cfoutput query="qGet">
        <input type="radio" name="regid" value="#registrationid#">#strname#<br>
    </cfoutput>
    <br />
    
    <strong>Message:</strong><br />
    <input name="strmsg" type="text" /><br /><br />
    <input type="submit" value="submit" name="submit">
</form>

As you can see, we get everything from the gcmtest table and output it with radio buttons and a message box. When you hit send it self posts and the cfhttp takes over. For the cfhttp we use a json struct containing the registration id and the message. This we pass to the url https://android.googleapis.com/gcm/send obviously not forgetting to pass along our authorization key we created in step one.

That's it for the server side!

Step 3 - Android In Action

Now we need to have our Android app a) register with and b) receive messages from CGM. Now there is a lot of code here, so I'll post the project on gitHub but I'm hoping to cover the basics. There are obviously loads of things you can do with this message but for now I'm just going to use the basic Google demo method of posting a notification. To do this we register a WakefulBroadcastReceiver which will keep the device alive in case it hears a broadcast message. Then it will fire an intentService.

a) Manifest.xml
You need these permissions:

    <!-- GCM connects to Google Services. -->
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.GET_ACCOUNTS" />
    <uses-permission android:name="android.permission.WAKE_LOCK" />
    <uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />

    <permission android:name="com.google.android.gcm.demo.app.permission.C2D_MESSAGE" android:protectionLevel="signature" />
    <uses-permission android:name="com.google.android.gcm.demo.app.permission.C2D_MESSAGE" />
    <uses-permission android:name="com.google.android.c2dm.permission.RECEIVE" />

You also need to register the services:

<receiver
      android:name=".GcmBroadcastReceiver"
      android:permission="com.google.android.c2dm.permission.SEND" >
          <intent-filter>
               <!-- Receives the actual messages. -->
               <action android:name="com.google.android.c2dm.intent.RECEIVE" />
               <category android:name="com.google.android.gcm.demo.app" />
          </intent-filter>
</receiver>
<service android:name=".GcmIntentService" />

Then what we're going to do in ActivityMain is check for the existence of a stored preference for the users name. If found we'll start a Fragment to welcome the user, if not we'll ask the user for their name.

FragmentTransaction ft = getSupportFragmentManager().beginTransaction();
String strName = getNameFromPrefs();

if(strName.length() > 0){
 Bundle bundle = new Bundle();
 bundle.putString(FragmentWelcome.TAG_NAME, strName);
 FragmentWelcome frag = new FragmentWelcome();
 frag.setArguments(bundle);
 ft.replace(R.id.activity_default_fragment_container, frag, FragmentWelcome.class.getSimpleName());
}else{
 ft.replace(R.id.activity_default_fragment_container, new FragmentName(), FragmentName.class.getSimpleName());
}

ft.commit();

Now FragmentName is really simple and I shall spare you the details. It is a simple layout with an input text box. It checks for internet and validates the input and if so it calls a listener which returns to the main activity. The Activity saves the user's name in the savedPreferences and opens FragmentWelcome.

FragmentWelcome covers the following steps:

• Get user's name via intent
• verify GooglePlayServices
• Gets your GCM Sender ID (created in step 1). Store this in strings.xml or somewhere sensible.
• Using your GCM Sender ID it posts to Google Play Services and gets your device id.
  

      private void registerInBackground() {
          new AsyncTask<String, Void, String>() {
              @Override
              protected String doInBackground(String... params) {
                  String msg = "";
                  String name = "";
                  try {
                      if (gcm == null) {
                          gcm = GoogleCloudMessaging.getInstance(context);
                      }
                      regid = gcm.register(SENDER_ID);
  

  
  
• Gets the URL of the receieve.cfm file we created in step 2 (again store it in strings or prefs) and appends the user's name and the device id to the url.
• Call an AsyncTask to open DefaultHttpClient and hit the url we have now created.
  

      private class UploadRegistrationID extends AsyncTask<String, Void, String> {
          @Override
          protected String doInBackground(String... urls) {
              String response = "";
              for (String url : urls) {
                  DefaultHttpClient client = new DefaultHttpClient();
                  HttpGet httpGet = new HttpGet(url);
                  try {
                      HttpResponse execute = client.execute(httpGet);
                      InputStream content = execute.getEntity().getContent();
      
                      BufferedReader buffer = new BufferedReader(new InputStreamReader(content));
                      String s = "ServerResponse:";
                      while ((s = buffer.readLine()) != null) {
                          response += s;
                      }
                  } catch (Exception e) {
                      e.printStackTrace();
                  }
              }
              return response;
          }
  

That's it. Once you've run your Android app and entered your name you should be able to go to your send.cfm file in a web browser and send a message through to your device. The message will then pop up in your notification window!

Here's the Android project on github:
https://github.com/jamessolo12/gcmPushDemo

Magic ;)

ColdFusion Railo deployment with Jelastic

I thought I'd try one of the cloud java hosting platforms out there and I must say I'm delighted I did. I wanted to start up a Jelastic instance and throw Railo on it and see how it worked out.

First up sign in is super simple, just your email and thats all you need. The user interface is clean, powerful and incredibly simple. With simple drop downs to create your environment and configure the number of instances etc that you need. In just a few minutes you can be up and running.
So i chose Tomcat 6. I did initally try Tomcat 7 but apparently they have a few problems with 7 at the moment. Not to worry 6 is fine. Deploy your environment and then wait for it to be deployed. This takes just a minute or so while they build your instance.

Then down at the bottom you'll see the deployment manager tab, under that you should see upload. You'll need to upload your Railo.war you can upload one you've downloaded, or I believe, upload direct from www.getrailo.org:

Once you've uploaded the .war file, you need to deploy it. Simple, still in the deployment manager tab, click on the box dropdown and click deploy. It'll ask you to confirm "ROOT" as the context, but root is fine. Once this is done you should be able to click the "launch in browser" button and see the railo admin show.

That's it, you're basically done. I expect you want your own application to run, but that's just as easy. If you click on the spanner / config option next to Tomcat 6 a settings tab will open and you can tweak the tomcat settings. Expand webapps and root. This is your application home, you can delete everything in there except for the WEB-INF folder. Then upload your cfm files and you're done!

I think this is a brilliant hosting environment and so brethlessly simple I'm very impressed.

Setup an AWS EC2 Instance Running Railo

Setup an AWS EC2 Instance Running Railo

OK so you want to delve deep into cloud computing and start your own instance Amazon Web Services (AWS) Elastic Cloud Compute (EC2) instance? Sadly the “official” Railo AMI seems to have died, so here we’ll be starting our own new one. We’ll be running firmly within the free criteria here and choosing options appropriately, most significantly this means linux! We’ll also be using putty to connect to our instance. For part of this tutorial we’re going to be running alongside the official Amazon starting an instance guide. So I will skimp on the details already covered by Amazon themselves:

http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/GetStartedLinux.html

Start-up an Instance

1. Click the giant “Launch an Instance” button. Select Basic Amazon Linux AMI I choose 32bit because it’s cheaper and (at the moment) free.
2. Ensure you’ve selected a micro instance.
3. Skip the instance details section, just accepting the defaults.
4. Create and download a Key Pair. This is important as it allows us to log onto our instance securely.
5. Next is the firewall or security groups section. This bit is important as it configures what applications and ports are allowed to access your instance. Create a new security group. You should select SSH and HTTP as a minimum, you can accept the default of 0.0.0.0 but that allows any IP access to these ports. This is fine for HTTP but if your ISP has given you a static IP then put this in for SSH.
6. Done, your instance will begin powering up. Watch the instances dashboard to see it’s status, eventually the status will flick to green, display “running” and the status checks will show 2/2. Once that happens we’re ready to logon.

Connect to your instance
I’m going to leave this bit a little to Amazon to explain. You’ll need to download and install putty and convert your key pair file (from step 4 above) into a putty private key file. Then grab your amazon public dns value (something like ec2-11-11-111-111.compute-1.amazonaws.com) and connect to it using putty. Don’t forget to enter the username ec2-user.

Get Linux straight & Install Railo

1. First lets get Linux to update itself:

sudo yum update

2. Download Railo to the instance:

wget http://www.getrailo.org/down.cfm?item=/railo/remote/download/3.3.1.000/tomcat/linux/railo-3.3.1.000-pl1-linux-installer.run

3. Assign permissions

sudo chmod 777 railo-3.3.1.000-pl1-linux-installer.run

4. Run Railo

sudo ./railo-3.3.1.000-pl1-linux-installer.run

Here you’ll want to accept all the defaults except three. First change the default password to something good. Second you should set the port to 80 (not 8888). This will allow normal connections to your server and links into why we had to allow HTTP (port 80) in the EC2 security group. Lastly you should say no to the apache connectors. This sets up railo with tomcat and installs tomcat for you. Of course if you’re more familiar with apache then go with that.

Hit your url
http://ec2-11-11-111-111.compute-1.amazonaws.com

The above link (customized for your public DNS) should show you the default Railo welcome page.

Use putty and VI to change your cfm files

1. Navigate to the webroot:

cd /opt/railo/tomcat/webapps/ROOT/

2. Remove all these files

sudo rm -rf *

3. Create a new index.cfm file

sudo vi index.cfm

VI - Linux editor

1. In vi to delete the contents of the whole file type

:1,$d

2. To swap between command and insert modes just press escape


3. To exit without saving, switch to command mode and press

:q!

4. To exit and save switch to command mode and press

:x

ColdFusion and Amazon AWS SES - Simple Email Service

Amazon Web Services (AWS) have a service called Simple Email Service (SES), SES is an excellent e-mail sending service for businesses and developers. It enables developers to quickly and easily send emails without having to go through the pains of setting up an SMTP server or any of that nonsense. It is intended for bulk sending marketing material, but you can also use it in very low volumes and it becomes a great tool.

To demonstrate the service, we’re going to use SES to send an email to ourselves using coldFusion. First and foremost you’ll need to sign up to SES (https://aws.amazon.com/ses/) and verify an email address. SES starts up in “development mode” where you have to verify every email address you want to send mail too. This is a fantastic feature that prevents accidental sending of email and lets you test out the functionality. For the purpose of this example development mode is perfect, so sign up and verify your own email address.

There are quite a few ways to get SES to send an email for you, we’re going to use what I think is the easiest to get started with, a POST request. It’s basically just liked submitting a form, but we’re going to use cfhttp to mimic a form post. First though, there’s a few things we need to setup before we actually do any coding.

1) Security details
Obviously it would be a pretty crummy service if anyone could use your email service to send emails to anyone, spam would run rampant! So there a few security details we need to get from amazon to authenticate each SES request. I won’t go into too much detail but you’ll need an “Access Key ID” and a “Secret Access Key”. You get these from amazon by logging into your aws account and clicking security credentials from the top right drop down. Look at this page for a bit more info:
http://docs.amazonwebservices.com/ses/latest/DeveloperGuide/QueryInterface.Authentication.html#QueryInterface.Authentication.Signatures

2) HMAC - Hash-based Message Authentication Code
http://en.wikipedia.org/wiki/HMAC
To authenticate our message Amazon requires use of a security algorithm called HMAC. The theory being if we (us and Amazon) have a shared private key, we can encrypt a known variable and authenticate ourselves. This is a very very brief introduction if you’ve never come across HMAC. HMAC is a cryptography method of creating a hash using a secret key. Normally a hash takes any value and creates a fixed length string. For example, using the MD5 hash in CF:

 hash(‘all your base are belong to us’)  

would return
847DBEB849668D30722D8A67BCED1C59

With HMAC we introduce a secret key to add an extended element of cryptography and security to the proceedings. Here’s an example:
 

 toString(toBase64(HMAC_SHA1('password','all your base are belong to us')))  

z0A498Rx0aNzXNOwfbjSKkhBdbA=

HMAC can use MD5 or SHA1 and Amazon does support both, but we’re going to use SHA1. Now coldFusion doesn’t offer the HMAC function by default, but that’s not to say we can’t write or borrow our own..

 <cffunction name="HMAC_SHA1" returntype="binary" access="public" output="false" hint="I create an HMAC hashed string from a given message and secret key.">  
      <cfargument name="signKey" type="string" required="true" hint="Secret key with which to encrypt message" />  
      <cfargument name="signMessage" type="string" required="true" hint="Message you want to hash"/>  
      <cfset var jMsg = JavaCast("string",arguments.signMessage).getBytes("iso-8859-1") />  
      <cfset var jKey = JavaCast("string",arguments.signKey).getBytes("iso-8859-1") />  
      <cfset var key = createObject("java","javax.crypto.spec.SecretKeySpec") />  
      <cfset var mac = createObject("java","javax.crypto.Mac") />  
      <cfset key = key.init(jKey,"HmacSHA1") />  
      <cfset mac = mac.getInstance(key.getAlgorithm()) />  
      <cfset mac.init(key) />  
      <cfset mac.update(jMsg) />  
      <cfreturn mac.doFinal() />  
 </cffunction>  

3) Create your Amazon authenticated header

Next we want to create our HMAC hash to pass to Amazon, they require us to encrypt the date and time, so to do so looks a little like this:

 <cfset variables.dteOldDate = now() />  
 <cfset variables.strDateString = "#DateFormat(variables.dteOldDate, 'ddd, dd mmm yyyy')# #timeFormat(variables.dteOldDate,'HH:mm:ss')# GMT" />  
 <!--- encrypt date --->  
 <cfset variables.xxxHMac = toString(toBase64(HMAC_SHA1(“yoursecretaccesskeygoeshere”,variables.strDateString))) />  

Next we need to integrate our hashed date with a header called a X-Amzn-Authorization

 <cfset variables.strAmazonHeader = "AWS3-HTTPS AWSAccessKeyId=#youraccesskeyidgoeshere#, Algorithm=HmacSHA1, Signature=#xxxHMac#" />  

So obviously replace youraccesskeyidgoeshere with the Access Key you got from Amazon.


4) Make the httpd call

That’s the complicated bit done, now all we have to do is make the cfhttpd call.

 <cfhttp url="https://email.us-east-1.amazonaws.com/" method="post" result="result">  
      <cfhttpparam type="header"       name="X-Amzn-Authorization" value="#variables.strAmazonHeader#" />  
      <cfhttpparam type="header"       name="Date" value="#variables.strDateString#" />  
      <cfhttpparam type="formfield"      name="Action" value="SendEmail" />  
      <cfhttpparam type="formfield"      name="Destination.ToAddresses.member.1" value="#yourToEmailAddressGoesHere#" />  
      <cfhttpparam type="formfield"      name="Message.Body.Text.Data" value="#yourEmailContentGoesHere#" />  
      <cfhttpparam type="formfield"      name="Message.Subject.Data" value="#yourEmailSubjectGoesHere#" />  
      <cfhttpparam type="formfield"      name="Source" value="yourFromEmailGoesHere" />  
 </cfhttp>   

That's it, run your code and you should receive an email!

Creating an Amazon AWS S3 Static Website

OK this is going to be a fairly short and simple post, but I want to take you through the steps as it will lead nicely into my follow up post. Basically what we’re going to do is use Amazon Web Services (AWS) S3 to host our static website. This is a great feature of AWS whereby their cloud storage services can be used to host a website.

1. Create a bucket - Sign Up for AWS and goto the console. Then click the S3 tab at the top and click “Create Bucket” underneath the tab. Buckets are amazon’s storage containers, they hold all your files. You’ll want to choose a region closest to where your users will be. Your bucket name has to be unique, and obviously useful to you, in my next post I’ll talk more about how you need to link it to your domain name if you decide to use a custom domain for your bucket.

2. Upload your files - Click on your bucket and upload your files. CSS, HTML, Images etc. I like to have a separate bucket for my images and or you can use AWS CDN services, but that’s another story.

3. Setup the bucket as a website - Click on your bucket and click properties. Goto the website tab and click Enabled and set the index document to your index.htm file or whatever. Click save.

4. Set bucket permissions - Click the permissions tab and click “Add bucket Policy” copy the following text, replacing the name of your bucket with the words mybucketname, then click save.

{
 "Version":"2008-10-17",
 "Statement":[{
    "Sid":"PublicReadForGetBucketObjects",
       "Effect":"Allow",
      "Principal": {
           "AWS": "*"
        },
     "Action":["s3:GetObject"],
     "Resource":["arn:aws:s3:::mybucketname:/*"
     ]
   }
 ]
}



That’s it, your URL should be something like:
http://mybucketname.s3-website-region.amazonaws.com