Messing about with Tarzan and the Amazon AAWS web service

At the moment I building a couple of applications running of the Amazon AWS web services – in a nutshell e-commerce sites that utilize products available via Amazon or companies selling through Amazon and then let you check out using Amazon’s secure checkout (the site owner gets commission on each product sold).

The system is great, but it is huge and there are an awful of options. The documentation is good, but quite hard to navigate and sometimes quite confusing, basically it just boils down to reading the API Docs.

I’ve been using the Tarzan AWS framework / toolkit / library to deal with the hassle of connecting to and being authenticated by the Amazon system. Tarzan is a great system, but again it boils down to reading the API as there aren’t really any tutorials.

Finding the OfferListingId

Products sold via Amazon are identified by all kinds of data including ISBN codes, ASIN (Amazon ID numbers), OfferListingIds etc. I guess the trick is knowing which one to use in given circumstances.

One real gotcha that held me up (I had to post on the Tarzan group to find the answer) was that Tarzan is designed to use the OfferListingId of a product to identify it at key points (e.g. adding a product to the remote cart) – my understanding of this is that it is because every item has an OfferListingId – but not every number has an ASIN (this could be wrong and it could be because of another reason entirely).

My problem was that I could not find the OfferListingIds of the products I wanted to add to the cart, and so was trying to use the ASIN. After an age of debugging and trying to figure out what the hell was wrong, I looked at the actual Tarazn code and discovered that it was set up to only deal with OfferListingIds – so I changed it to work with ASINs and the rest of my code started working how it was supposed to (i.e. I could now add items to my remote cart).

Ryan Parman (the guy who wrote Tarzan) kindly explained that the OfferListingId is only contained in certain responses from Amazon – I had been requesting a result set which did not include the OfferListingId – problem solved. It turns out that the OfferListingId is returned if you request a ‘Large’ or ‘Offers’ response group, I had been using the ‘Medium’ group thinking that less data to work with would make my life easier. Silly me.

Serialize SimpleXML problems

When you add your first product to the remote Amazon shopping cart you have to use the create_cart() method. This creates a cart object and adds product(s) to it. Part of the response is a CartId which you need to use to access the cart again in future requests. The logical place for this is a Session.

My responses from Amazon via Tarzan pop out as SimpleXML objects, very handy and easy to work with. Unless you forget that PHP won’t let you serialize native objects… There are various solutions but my quick and dirty solution was to use type casting to change the SimpleXML object to a string e.g.


$response = $aaws->cart_create($item, $opt, AAWS_LOCALE_UK);
	 	
    if(isset($response->body->Cart->CartId)){

        $this->Session->write('Cart.CartId', (string)$response->body->Cart->CartId);
	$this->Session->write('Cart.URLEncodedHMAC', (string)$response->body->Cart->URLEncodedHMAC);
	$this->Session->write('Cart.CartItemId', (string)$response->body->Cart->CartItems->CartItem->CartItemId);
	 		
	 		
}

And it works like a treat.

(This is also worth a read: Serialize and Unserialize SimpleXML in php)

Two CakePHP behaviours to extend MeioUpload

I’ve been using the the Meio Upload Behaviour for a while as its a very handy piece of code, but as is so often the case it doesn’t do quite what I need. In the past I’ve got round this by hacking together a kind of supporting framework in app_controller and scattered about in my models. But the other day I came across the newest version of the code by Juan Basso on Github and decided to dump my mess of spaghetti code and start from scratch.

Aims

The idea is that each model can have its own uploads with their own defaults, for example:

  • Products might need to generate images at 3 different sizes with the smallest zoom cropped.
  • News items might have 2 image sizes but also need to upload a PDF press release.

All of the uploaded files can be viewed and managed centrally from the Upload model.

The changes I wanted to make were as follows:

  1. Use a single Uploads table attached to multiple models (using the Polymorphic behaviour)
  2. The ability to do multiple uploads at once
  3. Rename my uploaded files
  4. Have an UploadVariants model / table with meta information about the thumbnails
  5. Be able to use more of the PHPThumb image options and use them on a per thumbnail basis.

Rather than just take the behaviour and start writing on top of it, I decided to extend the original behaviour (yes you can do this in CakePHP – its just easy to get absorbed in the framework and forget it) and then create an additional behaviour that manages the multiple uploads.

The underlying requirements are quite simple:

(Note 09/07/2009 you don’t actually need the Polymorphic Behaviour, unless you want to do fancy things with your finds)

And my two new behaviours:

At the end of the article is a link to let you download a zipped working CakePHP app with everything in place

The models

First things first. The database tables on which my models are based.

/*
MySQL Backup
Source Host:           localhost
Source Server Version: 5.0.51b-community-nt
Source Database:       je_meio
Date:                  2009/06/29 15:39:15
*/

SET FOREIGN_KEY_CHECKS=0;
#----------------------------
# Table structure for products
#----------------------------
drop table if exists products;
CREATE TABLE `products` (
  `id` int(11) NOT NULL auto_increment,
  `title` varchar(255) default NULL,
  `created` datetime default NULL,
  `modified` datetime default NULL,
  `created_by` int(11) default NULL,
  `modified_by` int(11) default NULL,
  `created_name` varchar(255) default NULL,
  `modified_name` varchar(255) default NULL,
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;

#----------------------------
# Table structure for upload_variants
#----------------------------
drop table if exists upload_variants;
CREATE TABLE `upload_variants` (
  `id` int(11) NOT NULL auto_increment,
  `upload_id` int(11) default NULL,
  `variant` varchar(255) default NULL,
  `filename` varchar(255) default NULL,
  `quick_type` varchar(50) default NULL,
  `width` int(4) default NULL,
  `height` int(4) default NULL,
  `created` datetime default NULL,
  `modified` datetime default NULL,
  `created_by` int(11) default NULL,
  `modified_by` int(11) default NULL,
  `created_name` varchar(255) default NULL,
  `modified_name` varchar(255) default NULL,
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;

#----------------------------
# Table structure for uploads
#----------------------------
drop table if exists uploads;
CREATE TABLE `uploads` (
  `id` int(8) unsigned NOT NULL auto_increment,
  `class` varchar(255) default 'Upload',
  `foreign_id` int(11) default NULL,
  `alt` varchar(255) default NULL,
  `filename` varchar(255) default NULL,
  `dir` varchar(255) default NULL,
  `mimetype` varchar(255) default NULL,
  `quick_type` varchar(50) default NULL,
  `filesize` int(11) unsigned default NULL,
  `position` int(11) default '0',
  `height` int(11) default NULL,
  `width` int(11) default NULL,
  `created` datetime default NULL,
  `processed` tinyint(1) default '0',
  `modified` datetime default NULL,
  `created_by` int(11) default NULL,
  `modified_by` int(11) default NULL,
  `created_name` varchar(255) default NULL,
  `modified_name` varchar(255) default NULL,
  PRIMARY KEY  (`id`),
  KEY `class` (`class`,`foreign_id`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
 class Upload extends AppModel {

  var $name = 'Upload';

  //The Associations below have been created with all possible keys, those that are not needed can be removed
  var $hasMany = array(
    'UploadVariant' => array(
      'className' => 'UploadVariant',
      'foreignKey' => 'upload_id',
      'dependent' => true,
      'conditions' => '',
      'fields' => '',
      'order' => '',
      'limit' => '',
      'offset' => '',
      'exclusive' => '',
      'finderQuery' => '',
      'counterQuery' => ''
    )
  );
  
  var $actsAs = array(
    'Polymorphic',
     'JeMeioUpload' => array(
         'filename' => array(
          'dir' => 'files/uploads',
           'create_directory' => true,
           'max_size' => 2097152,
           'max_dimension' => 'w',
           'thumbnailQuality' => 90,
           'useImageMagick' => false,
           'imageMagickPath' => '/usr/bin/convert',
           'allowed_mime' => array( 'image/gif', 'image/jpeg', 'image/pjpeg', 'image/png'),
           'allowed_ext' => array('.jpg', '.jpeg', '.png', '.gif'),
           'thumbsizes' => array(
             'small'  => array('width' => 90, 'height' => 90),
             'medium' => array('width' => 220, 'height' => 220),
             'large'  => array('width' => 800, 'height' => 600)
           ),
           'default_class' => 'Upload',
           'random_filename' => true
         )
       )
     );

}

In the Upload model, just include the Polymorphic behaviour and JeMeioUpload instead of MeioUpload. You could actually use the same defaults as the MeioUpload behaviour itself but there are two additional defaults ‘default_class’ => ‘Upload’, and ‘random_filename’ => true. These both refer to renaming files. As the behaviour can be used to manage uploads from multiple models in a single table it allows you to set the default Upload model. You can also set whether or not you want the files to have the meaningful part replaced with a random string.

class UploadVariant extends AppModel {

  var $name = 'UploadVariant';

  //The Associations below have been created with all possible keys, those that are not needed can be removed
  var $belongsTo = array(
    'Upload' => array(
      'className' => 'Upload',
      'foreignKey' => 'upload_id',
      'conditions' => '',
      'fields' => '',
      'order' => ''
    )
  );

}

The UploadVariant Model is just as it was baked.

class Product extends AppModel {

  var $name = 'Product';
  
  var $validate = array(
    'title' => array('notempty')
  );

  var $hasMany = array(
        'Upload' => array(
            'className' => 'Upload',    
            'foreignKey' => 'foreign_id',
            'conditions' => array('Upload.class' => 'Product'),
            'dependent' => true,
      'order' => 'Upload.position ASC'
        ),
        
    );
    
    var $actsAs = array(
     'JeMeioUploadPolymorphic'
   );
    
   
  
   var $jeMeioUploadParams = array(
      'filename' => array(
        'dir' => 'files/uploads/',
        'create_directory' => true,
        'allowed_mime' => array('image/jpeg', 'image/pjpeg', 'image/png', 'image/gif'),
        'allowed_ext' => array('.jpg', '.jpeg', '.png', '.gif'),
        'thumbsizes' => array(
          'small'  => array('width'=>60, 'height'=>60, 'image_options' => array('zc' => 1)),
           'large'  => array('width'=>800, 'height'=>400, 'image_options' => array('zc' => 0)),
           'shop'  => array('width'=>265, 'height'=>265, 'image_options' => array('zc' => 0))
        ),
        'random_filename' => false
      )
    );

}

Product is a test model set up to demonstrate the multiple uploads and the Polymorphic association to the Upload model. In this case the meaningful part of the filenames will not be replaced with a random string and the ‘small’ thumbnail will be zoom cropped. To find out more about image_options you will need to check the PHPThumb component and the documentation PHPThumb itself (play about and experiment) – what you can and can’t do will also be effected by whether or not you have ImageMagick on your server.

The Controllers

You don’t need to make any changes to your controllers at all.

The Views

There is nothing complicated in the views at all. Just remember to set them up properly for uploads.

The uploads/add.ctp

create('Upload', array('type' => 'file'));?>
	
input('alt'); echo $form->input('filename', array('type' => 'file')); ?>
end('Submit');?>

The products/add.ctp

This view allows up to 3 uploads to be stored in the associated Upload model.

create('Product', array('type' => 'file'));?>
	
input('title'); echo $form->input('Upload.0.filename', array('type' => 'file')); echo $form->input('Upload.1.filename', array('type' => 'file')); echo $form->input('Upload.3.filename', array('type' => 'file')); ?>
end('Submit');?>

Summary

The one part I’m not terribly happy with at the moment is error messages returning to the views in the Polymorphic associations (e.g. Product example). At the moment if an image or upload can’t be processed then there is no warning or error message. At the moment this is compromise I’m willing to live with.

I could certainly build some kind of component scaffold or something to go in the controllers to return an error message but I think the gain in responsiveness would probably be outweighed by the added complexity. Right now I can’t figure out an elegant way to get error messages from back to the views – if anybody has any suggestions please let me know.
(Note if you are uploading straight into the Upload model then the error messages are unaffected)

There is also the problem of what do you do when in a situation like the following: If you add a new Product successfully but the associated upload has problems? Ideally you would want to redirect to the edit view for that Product but there seems to be no easy way of doing this from a Model / Behaviour.

One solution that I have implemented in the past is to create dedicated views for uploading – so if you do hack your hacks are easy to manage.

I’ve baked a quick and dirty demo that you can download here. Its so basic that it doesn’t even show the uploaded images!

Todo List

  • A helper to generate things like uploads fields complete with delete checkboxes and thumbnail images.
  • Deleting the via the Polymorphic association (don’t think this works right now)

Overload $this->tags & $this->map in Apphelper

CakePHP is full of amazingly handy little bits and pieces, the trick is discovering them by exploring the API or paying attention to people who know Cake inside out.

AppHelper sits there begging to be filled, the idea being that you can overload core helper methods here; however there is a lot of debate as to whether or not this is actually a good approach and whether it is infact better (safer?) to overload helpers on a case by case basis (e.g. MyFormhelper extends FormHelper). (Personally I think it would be great to have the core helpers moved into base helpers so you can overload the core methods and still refer to them easily (e.g. HtmlHelper extends BaseHtmlhelper extends AppHelper, where HtmlHelper becomes an empty Class)

Two of the things I always have set in AppHelper are as follows:

1. Setup $this->map how I like it. $map defined in the Formhelper (cake/libs/views/helpers/form.php) is part of the auto magic behind forms. When you create a form in a view, Cake queries the model schema and figures out what sort of field it should generate for each database field.

Whilst I can see the reasoning behind the way Cake generates all the <select> fields, I would much rather have Cake genearte a simple <text> field and then use a JavaScript date picker to populate the field.

If you take a morning or a day to mess about and customise your bake templates then you can save vast amounts of time – if you want a date field why not just have it come to life with all the hooks for your Javascript there already? (A good introduction for custom baking is ad7six’s post.)

2. Redefine Tags. One of my real annoyances (and I know this is silly) is the fact that when Cake creates hidden fields it does not flag them with any specific class. This means that styles applied to other <input> fields can easily cascade down and cause odd little lines to appear all over your forms. Often you wouldn’t even notice this, but once you change the background colour of your form… all hell can break loose.

Tags are defined in /cake/libs/view/helpers/html.php so a quick and dirty solution is just to wrap a div with class around the hidden inputs so they can easily be distinguised css wise from all the other tags.


class AppHelper extends Helper {

  /*
   * overload the default type mappings is /cake/libs/views/helpers/form.php FormHelper
   */
  
  var $map = array(
          'string'  => 'text',    
          'datetime'  => 'text',
          'boolean'  => 'checkbox',  
          'timestamp' => 'text',
          'text'    => 'textarea',  
          'time'    => 'time',
          'date'    => 'text', 
          'float' => 'text'
        );
        
		
   function __construct() {
        parent::__construct();
        
        /*
        * overload the default html tags /cake/libs/view/helpers/html.php
        */
        $this->tags['hidden'] = '
'; } }

If you are looking for other things to stick in your AppHelper then try Matt Curry’s app_helper url caching.

At last the CakePHP books have arrived

I’d like to say it never rains but it pours, but that would be entirely the wrong sentiment, I need something more like good things come to those who wait

Anyway after an epic wait a flurry of CakePHP books have arrived.

I haven’t read any of these books yet, but I from personal experience I would say the Apress books are bound to be good. I have no idea what the books from Packt are like. I have one Manning book which is good but is unfortunately a PDF (and I hate reading PDFs).