31 July 2013

Asus Zenbook with Ubuntu - part 2


Last time I blogged about this, I'd managed to dual boot Windows 8 and Ubuntu. Both were running off the large capacity HDD on my zenbook, Windows was using the 24Gb SSD as a cache drive. Ubuntu wasn't using it at all, and I noticed this in boot speeds. After a few weeks of this I hadn't booted into Windows 8 once so I decided to make the leap of faith and re-install Ubuntu over top of everything. Death to Windows 8!

I used the same method as before, go into UEFI, plug in the Ubuntu install key, change nothing and save and exit. Then immediately hold escape. This gave me the option to boot from USB and of course select install Ubuntu.

Ubuntu install started and I had to pick my language, wifi and a few other normal things. Then came the selection screen for where to install Ubuntu, this is the step I was more interested in. Ubuntu offered the choice to erase everything, or dual boot, etc etc. I was after "something else" down at the bottom. Basically the advanced setup. This allows you to customize where each piece of Ubuntu is installed, what extra partitions are setup and how.

This was quite advanced, a little tricky and quite a bit of background reading was required. This is a page I leaned heavily on for some advice:

http://askubuntu.com/questions/234111/how-to-boot-ubuntu-from-ssd-drive-which-cannot-be-selected-as-boot-device

I basically decided to install the OS on the SSD and install other crucial parts on the HDD. My reasoning being from what I've read and experienced, the SSD isn't bootable (which is lame). However booting off the HDD and then utilising the SSD for important file system reads works just as well.

To cut a long story short, after a considerable amount of time experimenting with partitions, I've now settled on, and have working the following partitions:

  • SSD - filesystem. Whole thing set up as /
  • HDD
    • 10Mb - BIOS Boot
    • 250Mb - EFI Boot
    • 200Mb - /Boot
    • 3Gb /Linux Swap file
    • Free space
That's about it, I had to repeat the method I did last time of finishing the install, then booting off the USB again, going into try Ubuntu and running boot repair. However that was easy this time and after that it all seemed to work!

Hope it's of some use.



29 July 2013

Android app using database sqlLite

I created a cute little test app today to try out a few database concepts on Android. Thankfully it's really easy and there are some great tutorials out there. Anyway some of the concepts are really useful so I thought I'd jot down my findings.

First I created my new Android project ignore the MainActivity for now. I create a pretty standard Muppet.java class. I shan't patronise you with the code, int ID, String name and getters and setters for each. Then I create a new class called MySqlLiteHelper.java this is what will do the db interaction for us.

MySqlLiteHelper.java:
import java.util.ArrayList;
import java.util.List;

import android.content.Context;
import android.database.Cursor;
import android.database.sqlite.SQLiteDatabase;
import android.database.sqlite.SQLiteOpenHelper;
import android.util.Log;

public class MySqlLiteHelper extends SQLiteOpenHelper {

 private static final String DATABASE_NAME  = "muppets.db";
 private static final int DATABASE_VERSION = 1;
 private static final String TABLE_NAME  = "theusers";

 // Database creation sql statement
 private static final String DATABASE_CREATE  = "create table " + TABLE_NAME + " (userid integer primary key autoincrement, muppetname text not null);";
 // DO NOT DO THIS IN YOUR CODE:
 private static final String DATABASE_ADD_USER  = "insert into "  + TABLE_NAME + " (muppetname) values (";
  
 
 public MySqlLiteHelper(Context context) {
  super(context, DATABASE_NAME, null, DATABASE_VERSION);
 }


 @Override
 public void onCreate(SQLiteDatabase database) {
  database.execSQL(DATABASE_CREATE);
 }
 

 @Override
 public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
  Log.w(MySqlLiteHelper.class.getName(), "Upgrading database from version " + oldVersion + " to " + newVersion + ", which will destroy all old data");
  db.execSQL("DROP TABLE IF EXISTS " + TABLE_NAME);
  onCreate(db);
 }
 
 
 public void addNewuser(SQLiteDatabase database, String name){
  // DO NOT DO INSERTS LIKE THIS IN YOUR CODE. FIND OUT WHY I THE NEXT ARTICLE.
  String insertString = DATABASE_ADD_USER + "'" + name + "');";
  database.execSQL(insertString);
 }
 
 
 public List getAllUsers(SQLiteDatabase database){
  List muppets = new ArrayList();
  String[] columns = {"userid","muppetname"};

  Cursor cursor = database.query(TABLE_NAME, columns, null, null, null, null, null);

  cursor.moveToLast();
  while (!cursor.isBeforeFirst()) {
   Muppet muppet = cursorToMuppet(cursor);
   muppets.add(muppet);
   cursor.moveToPrevious();
  }
  // Make sure to close the cursor
  cursor.close();
  return muppets;
 }
 
 
 private Muppet cursorToMuppet(Cursor cursor) {
  Muppet muppet = new Muppet();
  muppet.setMuppetId(cursor.getInt(0));
  muppet.setMuppetName(cursor.getString(1));
  return muppet;
 }
 
} 
  • MySqlLiteHelper - This function is the constructor, on initialization it will create the DB.
  • onCreate - This function created the database table by executing the static string at the top.
  • onUpgrade - I think this function is pretty clever, if you change the db version number, it'll self upgrade! Amazeballs!
  • addNewuser - This is an ugly way to insert a user, I'll change this later.
  • getAllUsers - Here we're creating a cursor and querying the db for all the users. Then we're reversing through the cursor and putting the user object into a list. I opted to reverse the list so new Muppets show up at the top. Although it's quite easy to traverse it normally and have new users at the bottom.
One quick thing that's important to mention is we're using a function in Muppet.java to force the fragment to show the muppet name, instead of the object id.

@Override
public String toString() {
 return name;
}


Our next step is to create the layout file. As an additional step I've included a fragment. This lets us compartmentalise our layout neatly. If you want to just dump a listView on the layout, you could also do that quite easily.
 <Button
  android:id="@+id/btnKermit"
  android:layout_width="wrap_content"
  android:layout_height="wrap_content"
  android:layout_alignParentLeft="true"
  android:text="Insert Kermit" />

 <Button
  android:id="@+id/btnFuzzy"
  android:layout_width="wrap_content"
  android:layout_height="wrap_content"
  android:layout_alignParentRight="true"
  android:text="Insert Fuzzy" />

 <Button
  android:id="@+id/btnRefresh"
  android:layout_width="wrap_content"
  android:layout_height="wrap_content"
  android:layout_below="@+id/btnKermit"
  android:layout_centerHorizontal="true"
  android:layout_marginTop="19dp"
  android:text="Refresh" />
 
 <fragment
  android:id="@+id/fragment1"
  android:name="com.example.testdata.ShowFragment"
  android:layout_width="wrap_content"
  android:layout_height="wrap_content"
  android:layout_alignParentBottom="true"
  android:layout_below="@+id/btnRefresh"
  android:layout_centerHorizontal="true" />
Notice that this fragment is pointing directly to our ShowFragment.java class which we'll create now:
import java.util.List;

import android.database.sqlite.SQLiteDatabase;
import android.os.Bundle;
import android.view.View;
import android.widget.ArrayAdapter;
import android.widget.ListView;
import android.app.ListFragment;

public class ShowFragment extends ListFragment {

 private SQLiteDatabase database;
 private MySqlLiteHelper dbHelper;
 
 @Override
 public void onActivityCreated(Bundle savedInstanceState) {
  super.onActivityCreated(savedInstanceState);
  
  dbHelper = new MySqlLiteHelper(getActivity());
  database = dbHelper.getWritableDatabase();
  
  showAll();
 }
 
 public void showAll(){
  List values   = dbHelper.getAllUsers(database);
  final ArrayAdapter adapter = new ArrayAdapter(getActivity(), android.R.layout.simple_list_item_1, values);
  
  setListAdapter(adapter);
 }

 @Override
 public void onListItemClick(ListView l, View v, int position, long id) {
  // Do something with the data
 }
}
This should be pretty straightforward. OnActivityCreated we get the database instance, then call showAll. showAll() calls getAllUsers and puts the results in a List. This List is then converted to an adapter and we use setListAdapter to show the results in the fragment.

Bingo!
The main activity doesn't do anything special, it has a couple of buttons for adding new muppets, basic listeners which call
dbHelper.addNewuser(database, "Kermit");

Oh and there's a refresh button which updates the fragment:
 public void refresh(){
  //get fragment to refresh
  ShowFragment viewer = (ShowFragment) getFragmentManager().findFragmentById(R.id.fragment1);
     viewer.showAll();
 }

You're done! Super easy, hope it helps.





24 July 2013

Google Play Game Services


I've previously created my own custom built high scores add on for my Android app, frankly it was pretty rubbish. It was local only, not very pretty and had no public competition element. There are a few third party apps but they cost money. That is until at Google IO Google announced Google Play Game Services. I thought I'd take a look and I was incredibly impressed.

http://developer.android.com/google/play-services/games.html

Game Services allows you to instantly integrate a leaderboard into your app. It does this using Google + which means your users can share their scores with their circles, specific friends or of course with the public. I love this because just thinking about creating a user id for every user and a server side backend to deal with all that makes me shudder. Plus you can add in achievements which is pretty cool too.

So if you want to get started adding play services I very much recommend downloading Google's sample app "Type-a-Number":

https://developers.google.com/games/services/android/quickstart

Then run through the steps to get the API set-up on Google's developer console and of course provides a sample app and code to study. I really recommend you study this and get a good grasp on how it all works. I'm not going to run through all the code here, as Google have already done that. The same applies to the API, you do need to do a bit of setup on Google's developer console. Just make sure you copy the IDs of your game service and your leaderboard into your android project. Preferably in the same way Google suggest with an ids.xml file.

You're going to need to set-up eclipse and your project with Google Play Services Library which I already had for Google Maps anyway. The sample app uses the Base Game Utils library, which isn't totally necessary, but it does make things a little simpler. Again, all steps are covered in the quickstart.

The first thing is to get the Google Plus button working. This is a simple case of adding in the com.google.android.gms.common.SignInButton button to your layout file.
Then add a listener:

//Listener for Sign in
findViewById(R.id.sign_in_button).setOnClickListener(new View.OnClickListener() {
    public void onClick(View v) {
        beginUserInitiatedSignIn();
    }
});

Now beginUserInitiatedSignIn() is a function in Base Game Utils which deals with the Google + magic. It does have a callback function though which you can utilise, here's mine:

@Override
public void onSignInSucceeded() {
    Toast.makeText(this, "Signed in", Toast.LENGTH_SHORT).show();
    
    mShowSignIn = false;
    updateUi();

}

UpdateUi() handles hiding the sign in button and showing the sign out button.

Last but not least we need to push our scores.

getGamesClient().submitScore(getString(R.string.leaderboard_highscores),intScore);

It's spectacularly easy to do.

There are however a few things I've learnt:
  1. Be very wary of your certificate. You'll need to make sure the SHA1 hash you use is the same you export your Android project as the one you enter on the game services API.
  2. When you come to release this, if you've been using your debug certificate you'll need to delete your game services on the API and start it all over again. I've tried many times to tweak the SHA1 in an active game service but it always breaks. So best to start fresh with a production ready certificate and use it for the API as well.
  3. The IDs you get from play services must be copied exactly to your ids.xml file and your manifest must have a line like this:
    <meta-data android:name="com.google.android.gms.games.APP_ID" android:value="@string/app_id" />

  4. If you want to delete your scores or change the IDs you use for the game services or leaderboard you need to goto:
    Settings -> Google -> Google + -> Apps With Google + Sign In -> xx_AppName_xx -> Disconnect App
    You'll get the option here to delete your scores. This can be a bit temperamental, so change IDs only if you're really stuck.

One last thing worth mentioning, and its addressed to Google, whom I'm sure study my blog every day! When one looks at the achievements board, you can't expand the achievement to get the full description of what is required. On a phone, this means you'll never know what's required.

Thanks and good luck.

15 July 2013

Asus Zenbook and Ubuntu


I recently bought a shiny new Asus Zenbook, which which I'm delighted. However it came with Windows 8 and try as I might the sales guy wouldn't offer me anything else. I've recently blogged about my intense hatred for all things Windows 8 related. So I've decided to put Ubuntu on it, for now I'm dual booting. With past experience Ubuntu is so awesome, plus quick and easy to install I might decide to go back and clobber Windows 8 altogether.

The following are a few hints and tips I picked up if you try something similar. The bulk of the problem arises around the new version of BIOS called UEFI. This feels like an Intel thing that's been designed specifically for Windows and the rest of the world now has to catchup. Anyhow, I digress.
http://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface

The first thing I did was backup Windows 8. Just in case I needed to roll back. Then of course create a Ubuntu USB Install Key.

  1. Create a system image on a external hard drive or DVDs
    http://www.techrepublic.com/blog/window-on-windows/restore-windows-8-with-system-image-recovery/7464
  2. Create a Bootable USB Reovery Key
    http://www.techrepublic.com/blog/window-on-windows/create-a-recovery-drive-in-windows-8/7261

  3. Create a Bootable Ubuntu Install Key
    http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-windows

Thanks to Greg Shultz for those great articles.

That's the easy bit. In the good old days of BIOS that was basically it. You could insert your USB Ubuntu key and reboot, possibly tweaking BIOS boot order slightly. Sadly this is where UEFI makes things very difficult.

First I had to hold delete to get into UEFI which can easily be missed as the setup window gets smaller all the time. Then you need to disable fast boot and secure boot.Now plug in your Ubuntu USB key and save and restart. As soon as you're out of UEFI hold down the escape key. This brings up the boot options. Hopefully your USB key will be listed there. If not select "Enter Setup" and try again. Sometimes this takes a few goes.

Once you've got the USB Key to boot, you can install Ubuntu as a dual boot or "Full Windows annihilation" mode. This is all pretty self explanatory.

When you finish you'll be asked to reboot. When I did this I went straight back into Windows. Lame.

To combat this I went back into UEFI, didn't change anything, but did a save and exit (F10). Then again held the scape key and selected my USB key from the boot menu. Instead of install, this time select trial, or run from usb key. This gets you running a version of Ubuntu without installing anything. From there follow these instructions:
https://help.ubuntu.com/community/Boot-Repair

This will re-install the boot loader that windows 8 has interfered with. As I mentioned I did a dual boot, so it re-installed GRUB and allowed me to select my desired OS on boot.

That should be it. Remove your USB key and you should be running Ubuntu.

08 July 2013

Android Spinner With Linked Values

In all the languages I've used I've had need for a drop-down. However they all seem to call it something different. .net calls it a combo box, html calls it a select tag and Android appears to call it a Spinner. Oh well! Normally I've used it as a key value pair. That is to say the string you select, isn't normally the same as the value you pass in the result. For example

<select>
    <option value="42">Volvo</option>
    <option value="24">Ford</option>
</select>

Android makes this a little tricky as it doesn't offer the same functionality, but using a hashmap provides a good way to go keep a map of options and their corresponding value.


private String[]            arrMuppetNames  = {"Kermit","Gonzo","Fuzzy","Animal"};
HashMap<String, Integer>    hashMuppets     = new HashMap<String, Integer>();

Setting up the hashmap is fairly trivial using the put method. In this example I used a set of if statements with hard coded values. Normally you'd get your values from a web service or a db call.

One problem I encountered with this method is when you setup your OnItemSelectedListener it will get fired immediately when  you start your application. Technically this is correct as something has been selected onLoad, but not normally desirable activity. So to combat this I've added in a --please select-- value and ignored this element. This is fairly common practice in HTML and works in Android too.

So this is the function that builds the spinner, note we're creating a new array with the zero value set to --please select--.

private void setSpinnerContent(String arrMuppets[]){
    Spinner spinner            = (Spinner) findViewById(R.id.muppets_spinner);
    String arrNewArray[]    = new String[arrMuppets.length + 1];
    
    //Add please select to spinner
    arrNewArray[0]    = this.getString(R.string.select_please);
    for(int i=0; i < arrMuppets.length; i++){
        arrNewArray[i+1]    = arrMuppets[i];
    }
    
    //Application of the Array to the Spinner
    ArrayAdapter<String> spinnerArrayAdapter = new ArrayAdapter<String>(
            this,   
            android.R.layout.simple_spinner_item, 
            arrNewArray
    );
    spinnerArrayAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item); 
    spinner.setAdapter(spinnerArrayAdapter);
}

Then last of course is the selected item listener

private Spinner.OnItemSelectedListener spinnerListener = new Spinner.OnItemSelectedListener(){
    @Override
    public void onItemSelected(AdapterView<?> arg0, View arg1,int arg2, long arg3) {
        Spinner        my_spinner    = (Spinner) findViewById(R.id.muppets_spinner);
        
        //arg 3 is index, if 0 it means they clicked "--please select--"
        if(arg3 != 0){
            //get selected content
            String    strName        = my_spinner.getSelectedItem().toString();
            //use hasmap to translate that into a corresponding id. 
            int        intCatID    = hashMuppets.get(strName);
            
            Toast.makeText(getApplicationContext(), String.valueOf(intCatID), Toast.LENGTH_SHORT).show();
        }
    }
    @Override
    public void onNothingSelected(AdapterView<?> arg0) {
        //this is what happens if we select nothing.
        return;
    }
};

As you can see, we ignore the --please select-- and then call the hashmap to get the value.

Easy! Here's the full code:
http://pastebin.com/kVxwEHgm

01 July 2013

Amazon AWS - Signature Version 4


If you decide to try and interact with AWS Glacier API or certain other AWS services you will need to interact with their signature version 4 authentication. Unfortunately in ColdFusion this is one of the hardest things I've ever had to do. Not really ColdFusion's fault, and not really Amazon's fault. Their documentation is comprehensive (although a little confusing) it is just incredibly fiddly. Hashing is a process where a single wrong character completely changes everything. So one slip up causes failures and it can be difficult to determine what you've done wrong.

I have previously blogged about AWS Signiature Version 2 and using it with AWS SES. This article also touches on HMAC and a few of the other key concepts:
http://webdeveloperpadawan.blogspot.co.uk/2012/02/coldfusion-and-amazon-aws-ses-simple.html


AWS Glacier - http://aws.amazon.com/glacier/

AWS Glacier is a very low cost storage solution designed for archiving and backing up data. The basic idea is its cheaper than S3 storage but access is limited. So you don't necessarily have immediate access to your backups. Instead access can be requested and files retrieved within a given time period.
In order to keep costs low, Amazon Glacier is optimized for data that is infrequently accessed and for which retrieval times of several hours are suitable.
- AWS Website
In order to make an API request to Glacier you are required to authenticate each request using their V4 Signature process. Data in Glacier is stored in "Vaults", similar to an S3 bucket a vault is a storage container. For the purposes of this demo I've created a vault using the AWS web management interface. I will be using the API to list all available vaults. In time I hope to expand tutorials and code to cover more complex operations. However, once you've got the signature sorted that shouldn't be too hard.

Code Glorious Code

Again I just want to make the point that I'm just addressing the signature here. I hope to expand the CFC to better deal with making full requests. That will come in time though.

Setup
This should be fairly self explanatory. 
variables.dteNow                  = DateAdd("s", GetTimeZoneInfo().UTCTotalOffset, now());
variables.strPublicKey            = "publickey";
variables.strPrivateKey           = "secretkey";

variables.oAwsSig4                = createObject("component","lib.awsSig4").init(strSecretKey = variables.strPrivateKey);

//We need a few custom date formats
variables.strCanonicalDate        = variables.oAwsSig4.getCanonicalDateFormat(dteNow = variables.dteNow);
variables.strShortDate            = variables.oAwsSig4.getShortDateFormat(dteNow = variables.dteNow);


Step 1 - Create A Canonical Request

The canonical request is basically a standard way to describe the request you are making to AWS. Be that a GET, POST to Glacier or whatever.


This is the aws documentation pseudocode that describes what's happening:

CanonicalRequest =
  HTTPRequestMethod + '\n' +
  CanonicalURI + '\n' +
  CanonicalQueryString + '\n' +
  CanonicalHeaders + '\n' +
  SignedHeaders + '\n' +
  HexEncode(Hash(Payload))

Although I think that's slightly wrong. In my example1 (below), the second empty line is unexplained.

Here's the call I make to the method:

//step 1, create canonical request
strCanonical    = variables.oAwsSig4.createCanonical(
    strHTTPRequestMethod       = "GET",
    strCanonicalURI            = "/-/vaults",
    strCanonicalQueryString    = "",
    arrCanonicalHeaders        = ["date:#variables.strCanonicalDate#","host:glacier.us-east-1.amazonaws.com","x-amz-glacier-version:2012-06-01"],
    arrSignedHeaders           = ["date","host","x-amz-glacier-version"],
    strPayLoad                 = ""
);

You'll note the payload is empty, so the hash at the bottom is simply a SHA-256 hash of "".

The finished Canonical request should look like this.
Example1:
GET
/-/vaults

date:2013-06-26T13:07:03
host:glacier.us-east-1.amazonaws.com
x-amz-glacier-version:2012-06-01

date;host;x-amz-glacier-version
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Step 2 - Create A String To Sign

The string to sign is a little harder to explain out of context. It's basically the message we will hash which will authorize the request to AWS. It's a short and simple string with a very concise format.

The pseudocode is fine here and describes it quite well:
StringToSign  =
Algorithm + '\n' +
RequestDate + '\n' +
CredentialScope + '\n' +
HexEncode(Hash(CanonicalRequest))

Here's my function call:
//Step 2 - Create String To Sign    
strStringToSign    = variables.oAwsSig4.createStringToSign(
    strAlgorithm        = "AWS4-HMAC-SHA256",
    strRequestDate        = variables.oAwsSig4.getStringToSignDateFormat(dteNow = variables.dteNow),
    strCredentialScope    = strShortDate & "/us-east-1/glacier/aws4_request",
    strCanonicalRequest    = strCanonical
);


This is the finished string to sign:
AWS4-HMAC-SHA256
20130626T133038Z
20130626/us-east-1/glacier/aws4_request
6d26f46dbf5d48665e06f44a2f9a65368b3b8d9ef45638b1496fbbe6604ed9db

Step 3a - Calculate Signing Key

OK Now it gets complicated. This is where we start running things through the HMAC function and problems quickly occur. If you do it right though, it'll all come together. AWS do this all in one step, but I think its easier as two.

Here's the documentation pseudocode:

kSecret = Your AWS Secret Access Key
kDate = HMAC("AWS4" + kSecret, Date)
kRegion = HMAC(kDate, Region)
kService = HMAC(kRegion, Service)
kSigning = HMAC(kService, "aws4_request")

Here's my function call:
//create singing key
bSigningKey    = variables.oAwsSig4.createSigningKey(
    dateStamp    = strShortDate,
    regionName    = "us-east-1",
    serviceName    = "glacier"
);

and here's the function:
<cffunction name="createSigningKey" access="public" returnType="binary" output="false" hint="THIS WORKS DO NOT FUCK WITH IT.">
    <cfargument name="dateStamp"    type="string"    required="true" />
    <cfargument name="regionName"    type="string"    required="true" />
    <cfargument name="serviceName"    type="string"    required="true" />
    <cfscript>
        var kSecret     = JavaCast("string","AWS4" & variables.strSecretKey).getBytes("UTF8");
        var kDate       = HMAC_SHA256_bin(arguments.dateStamp, kSecret);
        var kRegion     = HMAC_SHA256_bin(arguments.regionName, kDate);
        var kService    = HMAC_SHA256_bin(arguments.serviceName, kRegion);
        var kSigning    = HMAC_SHA256_bin("aws4_request", kService);
        
        return kSigning;
    </cfscript>
</cffunction>

I know I've not included the function in the other steps, but I want to highlight the importance of two things:
  • kSecret is "AWS4" + Secret Key, then cast into bytes. This is a very important step and where I was going wrong for quite a while.
  • variables.strSecretKey is the secret key you get from AWS in your account section. It's obviously secret and shouldn't be disclosed to anyone. In my example it's set in the variables scope of the cfc.
  • The function I use HMAC_SHA256_bin accepts a binary argument as param1. This is different from the example in my previous blog post on signature version 2, which used two strings as arguments.
So you can see with four HMAC steps, the tiniest mistake means a totally different response. Obviously AWS will be doing the same thing on their end, so if the two don't match, your request won't get approved.

Step 3b - Sign it!

OK Now we bring it all together

signature = HexEncode(HMAC(derived-signing-key, string-to-sign))

We take the signing key from step 3a and the string to sign from step 2:

bSignature    = variables.oAwsSig4.HMAC_SHA256_bin(strStringToSign, bSigningKey);

Step 4 - Put it all together

Now we make our request:
<cfhttp method="GET" url="http://glacier.us-east-1.amazonaws.com/-/vaults">
    <cfhttpparam type="header"         name="Date" value="#variables.strCanonicalDate#">
    <cfhttpparam type="header"        name="x-amz-glacier-version" value="2012-06-01" />
    <cfhttpparam type="header"        name="Authorization" value="AWS4-HMAC-SHA256 Credential=#variables.strPublicKey#/#variables.strShortDate#/us-east-1/glacier/aws4_request,SignedHeaders=date;host;x-amz-glacier-version,Signature=#lcase(binaryEncode(bSignature, 'hex'))#" />
</cfhttp>

  • Note we didn't hex encode our bSigniature from before, so I do that in the value. Probably better done elsewhere, but I'll get to it.
  • Also note the public key. Again this comes from AWS management console.
  • Note the three different dates, one is the strCanonicalDate we created at the top, the other is the short date and finally the hard coded glacier version date.
That's it. That should work!  Obviously you need the cfc. Take a look below for that.


Advice

My advice to anyone attempting to do this in ColdFusion or any other language is as such:
  1. Baby Steps - The documentation is presented in steps. Get each step working perfectly before moving onto the next. They all rely on each other, so a mistake early on will just cascade and waste time later.
  2. Unit Tests - I'm a huge fan of unit tests anyway, but in this case they really helped. Setting up some great unit tests using AWS examples will help you define your input and your output and tweak your code until you get the response you're looking for.
  3. Check Responses - If you actually make a request to AWS they will tell you in the response what the problem is. Plus they'll tell you the expected canonical request and the expected string to sign. These can really help iron out any tiny discrepancies
  4. Try it in Java -  CFML Doesn't have a native HMAC function (pre CF10), and converting too and from byte arrays caused endless problems. So I just did a few piecemeal functions in Java and learned from that.

The CFC

Ah of course one final step. The cfc. As I've said I hope to improve it a great deal and perhaps open source it and put it on cflib. I've also got a bunch of unit tests I wrote which may help people improve it.
Right now I'm just pleased to have got it working and don't want to forget it all so it's going on the blog.


Thanks

I ended up not using the code, but some of Ben Nadel's stuff was really useful to understand. He's written a great cfc that I urge anyone using HMAC in CFML to consider:
http://www.bennadel.com/blog/2412-Crypto-cfc-For-Hmac-SHA1-Hmac-Sha256-and-Hmac-MD5-Code-Generation-In-ColdFusion.htm




<cfcomponent output="false">

<!---
<OWNER> = James Solo
<YEAR> = 2013

In the original BSD license, both occurrences of the phrase "COPYRIGHT HOLDERS AND CONTRIBUTORS" in the disclaimer read "REGENTS AND CONTRIBUTORS".

Here is the license template:

Copyright (c) 2013, James Solo
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--->

    <cffunction name="init" access="public" returnType="awsSig4" output="false">
        <cfargument name="strSecretKey" type="string" required="true" />

        <cfset variables.strSecretKey        = arguments.strSecretKey />
        <cfset variables.strPublicKey        = "mypublickey" />
        <cfreturn this>
    </cffunction>


    <cffunction name="createCanonical" access="public" returnType="string" output="false" hint="Create the canonical request">
        <cfargument name="strHTTPRequestMethod"        type="string"    required="true"                 />
        <cfargument name="strCanonicalURI"            type="string"    required="true"                 />
        <cfargument name="strCanonicalQueryString"    type="string"    required="false"    default=""    />
        <cfargument name="arrCanonicalHeaders"         type="array"    required="true"                    />
        <cfargument name="arrSignedHeaders"            type="array"    required="true"                    />
        <cfargument name="strPayload"                type="string"    required="false"    default=""    />
        
        <cfscript>
            var intCount            = 0;
            var strHeaderString        = "";
            var strCanonicalRequest = 
                arguments.strHTTPRequestMethod        & chr(010) &
                arguments.strCanonicalURI            & chr(010) &
                arguments.strCanonicalQueryString    & chr(010);
            
            //Headers
            for(intCount=1; intCount <= arraylen(arrCanonicalHeaders); intCount++){
                strCanonicalRequest    &= arguments.arrCanonicalHeaders[intCount] & chr(010);
            }
            
            strCanonicalRequest    &= chr(010);
            
            //Signed headers
            for(intCount=1; intCount <= arraylen(arrSignedHeaders); intCount++){
                strHeaderString        = arguments.arrSignedHeaders[intCount];
                strCanonicalRequest    &= strHeaderString;
                
                //put a semi-colon between headers, or a new line at end
                if(intCount EQ arraylen(arrSignedHeaders)){
                    strCanonicalRequest    &= chr(010);
                }else{
                    strCanonicalRequest    &= ";";
                }
            }
            
            strCanonicalRequest    &= lcase(hash(arguments.strPayload, "SHA-256"));
            
            return trim(strCanonicalRequest);
        </cfscript>
    </cffunction>


    <cffunction name="createStringToSign" access="public" returnType="string" output="false" hint="I create the string to sign">
        <cfargument name="strAlgorithm"            type="string" required="true" />
        <cfargument name="strRequestDate"        type="string" required="true" />
        <cfargument name="strCredentialScope"    type="string" required="true" />
        <cfargument name="strCanonicalRequest"    type="string" required="true" />
        
        <cfscript>
            var strStringToSign  =
                    arguments.strAlgorithm            & chr(010) &
                    arguments.strRequestDate        & chr(010) &
                    arguments.strCredentialScope    & chr(010) &
                    lcase(hash(arguments.strCanonicalRequest, "SHA-256"));
            
            return strStringToSign;
        </cfscript>
    </cffunction>


    <cffunction name="createSigningKey" access="public" returnType="binary" output="false" hint="THIS WORKS DO NOT FUCK WITH IT.">
        <cfargument name="dateStamp"    type="string"    required="true" />
        <cfargument name="regionName"    type="string"    required="true" />
        <cfargument name="serviceName"    type="string"    required="true" />
        <cfscript>
            var kSecret        = JavaCast("string","AWS4" & variables.strSecretKey).getBytes("UTF8");
            var kDate        = HMAC_SHA256_bin(arguments.dateStamp, kSecret);
            var kRegion        = HMAC_SHA256_bin(arguments.regionName, kDate);
            var kService    = HMAC_SHA256_bin(arguments.serviceName, kRegion);
            var kSigning    = HMAC_SHA256_bin("aws4_request", kService);
            
            return kSigning;
        </cfscript>
    </cffunction>


    <cffunction name="HMAC_SHA256_bin" access="public" returntype="binary" output="false" hint="THIS WORKS DO NOT FUCK WITH IT."> 
        <cfargument name="signMessage"    type="string" required="true" />
        <cfargument name="signKey"        type="binary" required="true" /> 
        
        <cfset var jMsg = JavaCast("string",arguments.signMessage).getBytes("UTF8") /> 
        <cfset var jKey = arguments.signKey />
        
        <cfset var key = createObject("java","javax.crypto.spec.SecretKeySpec") /> 
        <cfset var mac = createObject("java","javax.crypto.Mac") /> 
        
        <cfset key = key.init(jKey,"HmacSHA256") /> 
        
        <cfset mac = mac.getInstance(key.getAlgorithm()) /> 
        <cfset mac.init(key) /> 
        <cfset mac.update(jMsg) /> 
        
        <cfreturn mac.doFinal() />
    </cffunction>


    <cffunction name="toHex" access="public" returnType="string" output="false" hint="I convert binary to hex">
        <cfargument name="bSignature"        type="binary" required="true" /> 
        <cfreturn lcase(binaryEncode(arguments.bSignature, "hex")) />
    </cffunction>
    

    <cffunction name="getCanonicalDateFormat" access="public" returnType="string" output="false" hint="I return a formatted date time for the canonical part of the process">
        <cfargument name="dteNow"    type="date"    required="true" />
        
        <cfreturn "#dateformat(arguments.dteNow, 'yyyy-mm-dd')#T#TimeFormat(arguments.dteNow, 'HH:mm:ss')#" />
    </cffunction>
    

    <cffunction name="getStringToSignDateFormat" access="public" returnType="string" output="false" hint="I return a formatted date time for the string to sign section">
        <cfargument name="dteNow"    type="date"    required="true" />
        
        <cfreturn "#dateformat(arguments.dteNow, 'yyyymmdd')#T#TimeFormat(arguments.dteNow, 'HHmmss')#Z" />
    </cffunction>
    

    <cffunction name="getShortDateFormat" access="public" returnType="string" output="false" hint="I return a short date time">
        <cfargument name="dteNow"    type="date"    required="true" />
        
        <cfreturn "#dateformat(arguments.dteNow, 'yyyymmdd')#" />
    </cffunction>


</cfcomponent>