20 March 2013

Hamlet's Monkey - Part 3

I've previously blogged about my Hamlet's Monkey project.

Part 1 where I introduced the concept and did it in CFML:
http://webdeveloperpadawan.blogspot.co.uk/2012/10/hamlets-monkey-code-for-fun.html

Part 2 Where I ported the project into a java class with some JSON file read and writing:
http://webdeveloperpadawan.blogspot.co.uk/2013/03/hamlets-monkey-part-2.html

OK So now this whole project is moving toward where I was really excited to take it. GOOGLE! hahah I want to put this project on Google App Engine (GAE) so it uses cloud computing. That way a GAE cloud instance is like an actual monkey, tapping away at the keyboard and trying to re-write Hamlet! Awesome! This straight away introduces two potential problems:
  1. I've got to turn the project into a java servlet
  2. Tracking progress - In part 2 we added file storage, this won't work in the cloud, so we'll need to find a better method.


OK So converting the main crux of the method to a servlet isn't that complicated. The first thing you need to do is install the GAE plugin for eclipse. https://developers.google.com/appengine/docs/java/tools/eclipse Then we'll convert the project we made in Part 2 to a servlet, the public class needs to extend HTTPServlet:
public class Shakespearemonkey extends HttpServlet {

and the main method becomes doGet, which is what is called when the servlet responds to a http get request:
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException {


    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException {
        int            intCount                 = 0;
        String        strSubString            = "";
        String        strShakespeare            = Shakespeare.replaceAll("[^a-zA-Z]", ""); //Shakespeare without spaces etc
        String        strMonkeyString            = "";
        String        strBestSoFar            = "";
        response.setContentType("text/html");
        
        while (intCount < intNumLoops) {
            intCount++;
            
            //generate the random guess, one keystroke at a time
            strMonkeyString    = strMonkeyString + Character.toString(generateRandomLetter());
            
            strSubString    =    strShakespeare.substring(0,strMonkeyString.length());
            
            //check if we've guessed correctly so far
            if(strMonkeyString.equalsIgnoreCase(strSubString)){
                //Is this our best guess so far
                if(strSubString.length() > strBestSoFar.length()){
                    strBestSoFar    =    strSubString;
                }
            }else{
                //incorrect guess, start again
                strMonkeyString    = "";
            }
        }
        
        trackProgress(intCount,strBestSoFar);
        
        //purely for output, re-read the latest and update user on progress
        try {
            response.getWriter().println("Good Morning, I am your monkey! I will be trying to guess the string: " + strShakespeare + "<br />");
            MonkeyResults monkeyresults    = readResultsFromFile();
            response.getWriter().println("My best guess so far is: ");
            response.getWriter().println((String) monkeyresults.getBestGuess());
            response.getWriter().println("<br />I have made ");
            response.getWriter().println((int) monkeyresults.getKeyStrokes());
            response.getWriter().println("keystrokes");
        } catch (IOException e) {
            e.printStackTrace();
        }
        
    }


So what I've done is created an object for storing all our progress data. Possibly not necessary / overkill but its OO and it feels good. It's just two getters and setters, so I won't bore you with the code. What is different in the trackProgress function though is I've replaced the JSON code with this:
        MonkeyResults        monkeyResults            = readResultsFromFile();
        
        if(monkeyResults.getKeyStrokes() != 0 && monkeyResults.getBestGuess() != ""){
            strBestGuessFromFile    = (String) monkeyResults.getBestGuess();
            intKeyStrokesFromFile    = (int) monkeyResults.getKeyStrokes();
        }


So I'm going to use the GAE datastore to keep track of our progress. The first step is writing to the datastore:

    static Key            theResultsKey    = KeyFactory.createKey("Results","tblMonkeyResults");
    DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
    

    
    private void writeResultsToFile(int intKeyStrokes, String strBestGuess){
        //I write the number of keystrokes and the best guess so far to the datastore
        Entity objDaoOut = new Entity("Results", "tblMonkeyResults");
        
        objDaoOut.setProperty("intKeyStrokes",intKeyStrokes);
        objDaoOut.setProperty("strBestGuess", strBestGuess);

        datastore.put(objDaoOut);
    }
The next of course is the new read function, which returns an implementation of our local object:
    private MonkeyResults readResultsFromFile() {
        //I read results from the datastore and return an instance of class MonkeyResults
        MonkeyResults monkeyResults    = new MonkeyResults();
        long intStrokes    = 0;
        String strGuess    = "";
        
        try{
            Entity objDaoIn = datastore.get(theResultsKey);
        
            intStrokes    = (long) objDaoIn.getProperty("intKeyStrokes");
            strGuess    = (String) objDaoIn.getProperty("strBestGuess");
        }catch(EntityNotFoundException e){
            //e.printStackTrace();
        }finally{
            monkeyResults.setKeyStrokes((int)intStrokes);
            monkeyResults.setBestGuess(strGuess);
        }
        
        return monkeyResults;
    }
That's basically it. We have converted our java function to a Java Servlet and modified the file read and write to use the Google Datastore. Simples.

Once you're done, you should be able to test it locally, if it works you right click on the project and goto Google -> Deploy to App Engine. Hey presto google uploads it all for you and you should have a successfully running monkey!

Here's my monkey: http://shakespearmonkey.appspot.com/ http://pastebin.com/1etNP1ge

17 March 2013

Hamlet's Monkey - Part 2


OK so a while back I started work on my oh so awesome Hamlet's Monkey project. Anyway, I figured if the loop is too large, it'll just choke my CF instance, so this might be better handled as a java project.

I'm going to focus on the red light method as the green light method is just pretty simple. Basically, loop for a specified number of times grab a random letter and then check it against the Hamlet string. If we're successful add a new letter, if not, the monkey has to start again!
So here's the crux of it all:


private static char generateRandomLetter(){
 Random rnd = new Random();
 char strChar = lstAlphabet.charAt(rnd.nextInt(lstAlphabet.length()));
 return strChar;
}

public static void main(String[] args) {
    int            intCount                 = 0;
    String        strSubString            = "";
    String        strShakespeare            = Shakespeare.replaceAll("[^a-zA-Z]", ""); //Shakespeare without spaces etc
    String        strMonkeyString            = "";
    String        strBestSoFar            = "";
    
    while (intCount < intNumLoops) {
        intCount++;
        
        //generate the random guess, one keystroke at a time
        strMonkeyString    = strMonkeyString + Character.toString(generateRandomLetter());
        
        strSubString    =    strShakespeare.substring(0,strMonkeyString.length());
        
        System.out.print(strMonkeyString + "==" + strSubString);

        //check if we're done
        if(strMonkeyString.equalsIgnoreCase(strSubString)){
            System.out.println(" ** Nice guess, move along please.");
            if(strSubString.length() > strBestSoFar.length()){
                strBestSoFar    =    strSubString;
            }
        }else{
            System.out.println("");
            strMonkeyString    = "";
        }
    }
    
    trackProgress(intCount,strBestSoFar);
    
}


What was more interesting was how to store the results. What I decided to do was to write to a file using a JSON struct. This file could then store the results which could be read in every time the class was run and we'll know its all time success rate.
So again, not a complicated couple of functions. trackProgress gets called from the main function. It just reads the results from the file, and tries to decode the two JSON variables into local variables. If the current best guess is better than the one on file, we use the new one. Otherwise we just add a running total of the number of loops we've done. The read and write functions should be pretty self explanatory.

        @SuppressWarnings("unchecked")
        private static void writeResultsToFile(long intKeyStrokes, String strBestGuess){
            
            //create json object
            JSONObject obj = new JSONObject();
            obj.put("intKeyStrokes", intKeyStrokes);
            obj.put("strBestGuess", strBestGuess);
            
            //write it to file
            try {
                FileWriter file = new FileWriter(strFileName);
                file.write(obj.toJSONString());
                file.flush();
                file.close();
         
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        
        private static JSONObject readResultsFromFile(){
            JSONParser parser = new JSONParser();
            JSONObject jsonObject = new JSONObject();
            
            try {
                //read from file
                Object obj = parser.parse(new FileReader(strFileName));
                jsonObject = (JSONObject) obj;
         
            } catch (FileNotFoundException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            } catch (ParseException e) {
                e.printStackTrace();
            }
            return jsonObject;
        }
        
        
        private static void trackProgress(int intKeyStrokes, String strBestGuess){
            JSONObject    jsonObject                = readResultsFromFile();
            String        strBestGuessFromFile    = "";
            long        intKeyStrokesFromFile    = 0;
            String        strNewBestGuess            = "";
            long        intNewStrokes            = 0;
            
            if(jsonObject.get("intKeyStrokes") != null && jsonObject.get("strBestGuess") != null){
                strBestGuessFromFile    = (String) jsonObject.get("strBestGuess");
                intKeyStrokesFromFile    = (long) jsonObject.get("intKeyStrokes");
            }
     
            //whichever string is longer, this is the new best guess
            if(strBestGuess.length() > strBestGuessFromFile.length()){
                strNewBestGuess        = strBestGuess;
            }else{
                strNewBestGuess        = strBestGuessFromFile;
            }
            
            //number of key strokes is cumulative
            intNewStrokes            = intKeyStrokes + intKeyStrokesFromFile;
            
            writeResultsToFile(intNewStrokes,strNewBestGuess);
            System.out.println("Finishing - " + intNewStrokes + " keystrokes-" + strNewBestGuess);
        }
Oh yea, there's one thing. You'll need the the json simple project to use all this json goodness. https://code.google.com/p/json-simple/

To import this library into your project in eclipse, simply right click on the project go down to "Build Path" -> "Add External Archives..." then add the json-simple.1.1.1.jar Simple :)

Here's the full code: http://pastebin.com/h29t3bD7 Have fun!

06 March 2013

Install and setup of mysql on an amazon EC2 Instance.


Amazon RDS costs me quite a bit and I utilize about 1% of its true potential. That's not to say it isn't a great service (it totally is), I just figured I could do it cheaper by running mysql on the same instance as my web server. Might be cheaper, might not, I'll see. It certainly won't be as straightforward as RDS which is super simple.


--setup db
sudo yum install mysql
sudo yum install mysql-server
sudo yum install mysql-devel
sudo chgrp -R mysql /var/lib/mysql
sudo chmod -R 770 /var/lib/mysql
sudo service mysqld start
/usr/bin/mysqladmin -u root password yourpasswordhere

Please note, my password is actually not "yourpasswordhere". Neither should yours be ;)
Now we must login to mysql and create the db
mysql -u root -p


mysql> CREATE DATABASE xxxx;
mysql> exit

Again, don't call your db xxxx, unless it's some very hardcore stuff!
Now if you're doing this with Railo you can setup your datasource in railo server admin.
http://xx.xx.xx.xxx/railo-context/admin/server.cfm

That's it, you're all ready to go.

Hats off Sam Starling from whom I figured this out:
http://www.samstarling.co.uk/2010/10/installing-mysql-on-an-ec2-micro-instance/
I've said it before and I'll say it again, if only there was a decent GUI for connecting to mysql and <cough> ms sql. <sigh>

03 March 2013

Connect to EC2 Instance from unix

I've spent a few days slightly mystified by not being able to connect to my Amazon AWS EC2 instance from my Ubuntu machine. Sadly I've fixed it now, so I can't re-type the error. Slightly concerning you may scream, but I could connect from a windows machine and I don't have any visitors anyway...so meh!

Anyway after much researching I stumbled across this:


chmod 400 mykeyname.pem 

This worked a charm, I can now connect via ssh or the java browser util that Amazon provide. Awesome! If this is a new AWS security change or not I don't know. Anyway, hope this helps someone!

05 February 2013

Checking the hash of a file

You know when you download a file and you're informed what the hash is, so you can verify your download? We'll I've often seen that and have a working knowledge of hash functions but have never bothered to try it.

Anyway, I thought I'd give it a go, and I'm pleased to say its incredibly easy if you have a CF server kicking around:

<cffile action="read" file="#form.fileupload#" variable="variables.myFile" />
<cfoutput>#hash(variables.myFile,"#form.sel_hash#")#</cfoutput>


Here's a slightly more completed code fragment if it helps:
<div class="jumbotron">
    <h1>Hash Your File!</h1>
    <cfif structKeyExists(form, "fileupload")>
        <cffile action="read" file="#form.fileupload#" variable="variables.myFile" />
        <p class="alert alert-info"><cfoutput>#hash(variables.myFile,"#form.sel_hash#")#</cfoutput></p>
    <cfelse>
        <p class="lead">I help you calculate the hash of a file</p>
    </cfif>
    <form class=".form-horizontal" method="POST" enctype="multipart/form-data" action="test3.cfm">
        <select name="sel_hash">
            <option value="MD5">MD5</option>
            <option value="SHA">SHA</option>
            <option value="SHA-256">SHA-256</option>
            <option value="SHA-384">SHA-384</option>
            <option value="SHA-512">SHA-512</option>
        </select>
        <br />
        <div class="fileinputs">
            <input name="fileupload" type="file" />
        </div>
        
        <br /><br />
        <button type="submit" class="btn btn-primary">Submit</button>
    </form>
</div><!-- /jumbotron -->

30 November 2012

Why I'm done with Microsoft Operating Systems

**Warning rant alert**! I'm going to complain in this article, which is something I don't like doing, but I feel I must.

I'll preface this artcile by saying some of Microsoft's stuff does work. Windows Server is a good product, SQL Server is an phenomenal product that does you credit. The Xbox again works beautifully, I just don't understand why you can't apply the same standards to your desktop operating systems.

I've spent years supporting Microsoft's operating systems, literally years, in that time I've built hundreds of computers and I make my living in IT. I first started my love affair with Microsoft in the days of MS-DOS, since then I've used, abused and installed every major operating system they have released. So I flatter myself that I know the market and I've tried, I really have.

..but I'm sick of this crap from Microsoft. Windows 8 is diabolical.

You've had twenty years to get it right and you give me Windows 8. This is embarrassing, if I worked for Microsoft I'd quit out of shame! I'm glad they fired Steven Sinofsky, he deserved it. I haven't been this frustrated since my Scalextric got trodden on!

OK OK I get it, building an operating system is no easy task. There are infinite combinations of processor / motherboard / memory etc and you have to support drivers and every piece of software under the sun. I'll admit, probably not a lot of fun.
Yes, yes I know we're supposed to embrace change and be adaptable. That might apply to a the weather or a restaurant menu but this is an operating system. Do car manufacturers suddenly decide put the gear stick on the roof? My Mum and Dad have to use this thing, are you going to spend weeks on the phone explaining how they can find their files? No you're probably not. If I struggle to use an operating system, how do you expect "average Joe" to use it?

For goodness sake Google have built Android in four years, you've had thirty seven!
Windows XP was good. It worked, it had a few problems but it mostly did what I wanted. Windows Vista is an embarrassing, unstable, disgrace and I think you knew it. Windows 7, credit where it's due is pretty good. It's mostly stable, quite fast and the user interface isn't appalling. So build on that, don't give me something like Windows 8.

The main problem appears to be it's built for touch screens, why in planet earth you made this decision is beyond me. This is a desktop operating system. DESKTOP. As in mouse and keyboard. I don't have a touch screen monitor, neither do I want one. Does anyone really use a touch screen monitor for every day regular use?

OK So here are the main reasons I hate it:
  1. You've replaced the classic well known start button with a gimmicky looking sneeze of applications scattered across the screen. It looks more like a stack of building blocks a three year old put together. You created the start button and surprise, it works, stick with it.
  2. How do I close an app? Seriously?
  3. Internet Explorer is a festering turd of an application that's set back web development decades. This aside, I've now got two versions? One as an app and one on the desktop? How does that make sense?
  4. IE Tabs, I've been using this thing 3 days and I still can't figure out how to switch tabs? Why is this sort of thing hidden? Tabs were designed for quick switching between pages. You've just taken me back to the dark ages of multiple browsers running.
  5. What the hell is this right hand menu thing? I still haven't found a use for it other than just to hide the shut down button.
  6. My running apps are displayed how? By hovering over the left and then dragging down? Because just showing what I've got open needs to be hard?
  7. People - ok so I like that I can sign in with my "msn messenger / messenger / live messenger / skype" account, cool. Well done, this is a nice feature. The great big "people" app looks like it could be really good. A simple intuitive method of seeing all my friends and getting in touch. If it were implemented right....it's not. All I want to do is see a list of my friends who are online on msn. I want to send IMs to my BFFs about how much this product stinks. Unfortunately this task is almost impossible. I hover over this and drag that, it takes me hours to just get this simple view of who's online.
  8. Stability - This is the big one. After 37 years I expect your operating system to just work. No crashes, no hanging, no embarrassing blue screens of death. I had this brand new totally clean OS running for less than 4 hours and boom. My first total lock up.
    Several hard reboots later I thought I'd apply some windows updates. BOOM, failure two. After 20 minutes waiting windows update announced it had failed to update and it was going to roll everything back....time passes....more time passes. 40 minutes later we're back to square one. FFS.
Well done MS, this stupid product is going back to the store and I'm switching to Ubuntu. After twenty odd years I've given up on you. I'm sorry, I really really really didn't want it to come to this but I've just run out of patience.

22 November 2012

ColdFusion - Find common elements in two lists

I recently stumbled across an interesting programming question, how to find the duplicates in two lists? In this case I had two lists of email addresses and wanted to know which email addresses were in both lists. Sounds simple.

I'm curious to know what is the optimum approach, both in terms of simplicity and in terms of speed.

The first solution (compareLists) is the one I came up with, the second (listCommon) is one I adapted a little from Phillip's Coldfusion Blog.

<cffunction name="compareLists" access="public" returnType="string" output="false">
    <cfargument name="strList1" type="string" required="true" />
    <cfargument name="strList2" type="string" required="true" />

    <cfscript>
        var stuCompare        = {};
        var intCount        = 0;
        var strTempElement    = "";
        
        //loop through second list
        for(intCount=1; intCount LTE listLen(arguments.strList2); intCount++){
            strTempElement    = listGetAt(arguments.strList2,intCount);
            if(listContainsNoCase(arguments.strList1,strTempElement)){
                //Add key to struct
                stuCompare[strTempElement]    = "";
            }
        }
    
        return structKeyList(stuCompare);
    </cfscript>
</cffunction>

<cffunction name="listCommon" access="public" output="false" returnType="string">
    <cfargument name="strList1" type="string" required="true" />
    <cfargument name="strList2" type="string" required="true" />
     
    <cfset var arrList1 = ListToArray(arguments.strList1) />
    <cfset var arrList2 = ListToArray(arguments.strList2) />
     
    <cfset arrList1.retainAll(arrList2) />
     
    <!--- Return in list format --->
    <cfreturn ArrayToList(arrList1) />
</cffunction>

I was a little surprised you could use a java function like retainAll quite so nativity, but other than that it's fairly self explanatory.
So to time the methods, I took a leaf out of Ben Nadel's book and used getTickCount() to count the processing of 10,000 iterations of the above functions with two small lists.

Function Name Time (in milliseconds)
compareLists 849
listCommon 611
compareLists 352
listCommon 182
compareLists 232
listCommon 234
compareLists 208
listCommon 151
compareLists 80
listCommon 104

So as you can see, the listCommon method is much quicker. However it seems after the first few runs, caching kicks in and from then on the compareLists method remains very fast. I don't really want to get into caching, I'd rather focus on the differences between the two methods. Does anyone have any suggestions or input? Anyone got any examples of how they'd do this kind of thing in Java or possibly even something lower level?