Friday, May 25, 2012

SitePen dgrid From Html and Footers

I'm still fairly new with using Javascript, so what I'm showing today is a way, but perhaps not the best way.

And that way is turn a simple HTML table into something nice using SitePen's dgrid, and adding a footer showing totals.

There are some basics you might want to be familiar with first, like Dojo (1.7.2 at the time of writing), using AMD, and perhaps using the SitePen dgrid.

I was doing a screen that has a whole bunch of HTML tables on it. And I thought that the Claro theme looked so nice, but the dojoTabular style was so out of place, that it would be nice to have the data styled like it was in a dgrid. Turns out, it wasn't as hard as I thought it might be. dgrid provides an awesome class that will convert the basics for a HTML table to a dgrid, called dgrid/GridFromHtml.

Since this was simple tabled data, and I didn't feel like creating a store for every table, I stored the data as a JSON value in a hidden form variable. In my Javascript, I would load that data, create the grid from the HTML, and use renderArray to populate the data.

Here's a small sample of what I did.

This is the HTML, though I've not bothered to populate the hidden element with actual JSON data. Just make it an array of entries, where each entry has columns that will match the column headers in the table.

My Table

Id Name Value

For your convenience, a simple function to convert the table to a grid.
function loadTable(dataId, id, options) {
    var data = dom.byId(dataId);
    if (data) {
        data = data.value;
        
        data = JSON.parse(data);
        
        var grid = new GridFromHtml(options, id);
        grid.renderArray(data);
            
        return grid;
    }
    return null;
}

And when you're ready, call that function.
   var tableGrid = loadTable('tableData', 'tableGrid');

That was fairly painless.

Now for the fun part that I wanted to show off. Adding a footer to display a total. I'll reiterate that this is a way, not the best way.

The basic idea is to prepare an entry similar to the data entries, render a row, and put it in the footer. It uses the same basic formula of putting that data into a hidden HTML element, loading it, parsing it, and rendering it.

I've added an entry to store the totals data. Leave the first two columns blank, and populate the total in the value column.

My Table

....

Now turn on the footer, load the footer, render it, and resize so the grid body can apply the right styles and not overwrite the last row in the grid.

   var tableGrid = loadTable('tableData', 'tableGrid', {showFooter: true});

   var tableTotals = dom.byId('tableTotalsData');
   tableTotals = tableTotals.value;
   tableTotals = JSON.parse(tableTotals);

   var footer = tableGrid.renderRow(tableTotals, {});
   footer = put('div.dgrid-totals', footer);
   put(tableTotals.footerNode, footer);

   tableGrid.resize();

Now we just need a little CSS to make sure the cells in the footer are aligned with the main content, and take the scroll bar into account.

.dgrid-footer .dgrid-totals {
    margin-right: 17px;
}

I wrap the rendered footer row in a div.dgrid-totals element, because it may not be the only footer on the table, especially if you're using pagination instead of on demand.

And there you have it.  Simple table data looking sexy with the rest of your Dojo themed site.

Wednesday, May 23, 2012

Saving HABTM for existing records in CakePHP 2.1

This is more of a note for me, since I was doing it wrong, and I couldn't find a clear example in the CakePHP manuals.

The scenario is saving a hasAndBelongsToMany (HABTM) relationship for existing records, and requiring a uniqueness constraint on the join table.  The solution is to use saveAssociated(). The array to pass as the data should look something like this.

$data = array(
    'Model' => array('id' => 1),
    'AssociatedModel' => array(
        array('associated_model_id' => 1),
        array('associated_model_id' => 2)
    )
);

The AssociatedModel is actually the name of the hasAndBelongsToMany relationship found on the Model model, and will represent the actual join table, which will probably be called models_associated_models.

Friday, April 20, 2012

You Never Go Full Retard


To paraphrase a line from Tropic Thunder, I've gone full retard with my usage of Dojo widgets.

It's not retarded, though. It's one of the proper usages.

Dojo seems to have two ways of doing their widgets. The first, and original way is by putting mark up in the HTML to denote the type and any properties. This is parsed at runtime.

The second way is complete programmatic insertion. Your HTML is a series of div tags with ids, and everything else is created and inserted with Javascript.

Up until now, I've been doing a combo of the two. I'd rather generate regular HTML from my back end framework (CakePHP), and then in the Javascript, identify elements to become Dojo widgets by id. The trouble with that approach is that Dojo is unprepared for it, and doesn't replicate some of the properties on the native element (like name) on to the newly created Dijit HTML.

It'll only be a matter of time before I get sick of adding the Dojo data-dojo-type attributes to my CakePHP form elements, and create a DojoFormHelper to default them for particular types of elements.

Wednesday, February 1, 2012

Censored in a country near you

Censorship, social media, tailored search content and privacy policies shakier that an oppositions family planning policy.

It seems like a large bubble is moving across the land again, and this it's the issue of local law enforcement controlling social media content. Last time it was the Occupy Wall Street movement.

For me, this bubble started growing sometime last year (or was it the year before) with Stephen Conroy and the great Australian firewall. A proposed change to Australian ISPs, requiring that they block a specific black list of URLs, with very little transparency of what that black list actually was.

Then it morphed a little into an unwillingness by some Australian state governor generals to recognise a R18+ rating for games. You'd think Australians would be able to call a spade, a spade. But not if it's a spade of particular quality. Manufacturers have to remove a little of the shine, then push it to an MA15+ market, all the while, uneducated, gormless parents are buying these for little Johnny, who can't be much more than 13. Oh dear, I think I've blurred my analogy. Anyway, I think you get the idea. A game that would otherwise be classed as R18+ has a few graphics touched up, and ends up in the hands of a 13 year old, put there by the parents who should know better. I think it would be a different result if the game was actually recognised and labelled as R18+. But I digress.

More recently, we've had SOPA/PIPA. While they're not about censorship, they are about going the wrong way to control information on the internet. Some of the ideas put forward about blocking which sites should and should not be available for the American viewing public smell a lot like some of the ideas that Conroy had for the great Australia firewall. And if either one of those reasons resulted in an implementation of blocking URLs or whole sites with little review or recourse for action, you can be sure there will be a justification for the other reason soon enough.

Then we have Twitter, once free of censorship, now adjusting policy to cater for local law, and telling us that some censorship is better than no censorship. Well, except for the people actually getting censored, or the intended audience not being reached because of censorship by local law enforcement in that country. The small blessing is that, for those that care, there's an independent site listing the requested take down notices. The details of that escape me, somewhat. But I'm not overly put out by the Twitter stuff. While I have an account, I go through phases of usage, and in more recent times, that usage is way less that Facebook. And Facebook gets a 5 minute look in, perhaps once every two weeks.

I couldn't tell you what's going on with Facebook. Since their more recent UI change, doing what's hot, what's popular and apparently, what's relevant, I get the feeling I'm not seeing all that I should be seeing, and no longer trust Facebook to deliver posts from friends. I know there's a similar thing going on with Google and their searches, in general, but since most of my searching is for programming syntax and documentation, I welcome relevant hits into StackOverflow, any day of the week.

The, the latest act is Google, and in particular, Blogspot and the usage of it's TLDs. Once upon a time (say, last week), this blog was only found through reuben-in-rl.blogspot.com. Now, the TLD will change depending on what country you are viewing from. You can read a bit more on it here. So Americans and their aliens will still see .com, Australians will see .com.au, and other people viewing from other nations will see whatever TLD Blogspot has managed to secure for that country.

So why did that do that? So local law enforcement can block content without impacting what content gets seen from other countries. Pretty similar to what Twitter are doing.

There are a couple of interesting side effects.

The first is a loss of page rank for any engine that just looks at the URL. Any Tweet counts, Facebook Likes and Alex rankings just got wiped for any country outside the US. I'm tempted to put Google's +1 in that basket as well, but Blogspot supplies the canonical relationship in the header of all the pages, which points back to the .com version. I wonder if a +1 gets applied to the viewed URL or to the canonical URL. Usage of canonical leads me to the second point.

The second side effect is the usage of the canonical relationship in the header that points back to the .com version. This means, for most search engines that honor the canonical reference, only the .com version of the website will get indexed. And if local law enforcement in the US decides your content isn't fit for viewing there, then you're pretty much fucked for having it viewed anywhere else. I guess this was always the case in the old system, but it's probably the one inconsistency with the TLD change policy. Your content is good for indexing, until you piss off the US.

So, my original thought that sparked this entry was "how long until distributed peer to peer technologies are used to disseminate blog content into the ether?". Maybe your audience reaches a critical mass, and instead of have a hosted blog, you just publish an atom to a few, well known locations and assorted technologies (RSS, Usenet, Github, Wordpress, Craigs List, Gumtree, IRC logs, free CDNs and a whole bunch of torrent trackers that peddle plain text and a handful of supporting images). Consider what happened when attempts were made to shut down WikiLeaks.

It's not a small world, after all. There are just more assholes cramping your style.

Update: Google will look at the canonical link, if you don't specify a href. Even then, you might read this post about how that's not quite good enough, but it seems Google have updated processing so that even if you +1 an explicit URL, if it contains a canonical link, that will increment counters for other +1 buttons with different URLs, but the same canonical link. TL:DR. The Blogspot TLD change won't bork your +1's.

Monday, January 30, 2012

Refreshing a Dojo DataGrid

Here's a little trick for refreshing the contents of a Dojo DataGrid to the currently selected position. You may wish to do this if your data source is being updated by something else, and you need to force a refresh.

dojo.require("dojo.aspect");
var grid = dijit.byId("myGrid");
grid.store.close();
var handle = dojo.aspect.after(grid, "_onFetchComplete", function() {
    handle.remove();
    this.scrollToRow(this.selection.selectedIndex);
});
grid.sort();

This simplified example closes the store and calls sort() to force a refresh of the data.

In the normal course of processing, Dojo tries to get you back to where you were, but due to the clearing of the data, the scroller height is reduced, and when Dojo tries to set the scrollTop property of the grid div, it remains as it's reset value of 0.  Therefore, we use aspect.after() to set the row after the fetch from the sort has completed.

We don't need this happening every time data is fetched for the grid, so we record the aspect handle, and force the aspect function to remove itself from the chain, once it has been called.  Since the scrollToRow call is likely to fetch more data, we remove the aspect handle before calling it, so we don't have the aspect function called twice.

This was done using Dojo 1.7.1.

Friday, January 27, 2012

Fixed layouts for tables

I'm still fiddling around with Dojo grids. It's the Enhanced Grid in 1.7.1 at the moment, but most of the CSS comes from DataGrid anyway.

I came cross an interesting difference between Chrome, Firefox and IE with regards to how Dojo implement their grids.

To get grids to render quickly, Dojo like you to specify the width columns in the grid structure. This means they don't have to run any tricky rendering calculations, and their arrangement of tables nested in div tags works out nice and quick.

However, Chrome and Firefox have different ideas on how to render what Dojo has done to make use of this quick rendering.

Dojo does the following: Makes each table have a table-layout of fixed, give the table a width of 0 and explicitly specifies the width of each column in the TH and TD tags. That width is the width you specified in the grid structure.

Based on this, I expect the widths I've supplied to be the total widths of the columns. I also use the sum of these widths, plus a bit more for the vertical scroll bar for the node holding the grid.

Chrome does the following: Pretty much as expected, from a "setting up Dojo" point of view. Each column is as wide as I configured. However, from the CSS point of view, it's a bit strange. Dojo puts padding in the cells (5px each side) and there's a border as well (1px all around). So when you look at the Metrics tab in the developer tools, the actual width displayed is less than the what you put it. The box-sizing of the TH and TD elements is content-box. It looks like Chrome as reverse engineered the supplied width to fit with the content-box model. My rationalisation is that Chrome forces the TH and TD elements to be box-sizing: border-box, given a table-layout : fixed, but instead of just saying that, it changes the width to suit box-sizing: content-box. Well, it all looks good in Chrome, so what do I care?

A lot, because my clients aren't using Chrome. They're using Firefox and IE.

Firefox does the following: Completely ignores the width on the table (which is 0), in favour of the widths on the column headings. And then proceeds to render them using box-sizing: content-box. This means all the "exact" column widths I asked for are now increased by the padding and the borders in the column headings.

The specification for table-layout : fixed at W3C is particularly vague when it comes to determining what part column heading widths, and their paddings, should play when determining the total width of the table.

On one hand, Chrome seem to have taken their lead from the second paragraph, and have used the block width algorithm, to determine that a supplied width should be applied as though there was a box-sizing : border-box applied. Even then, that's not quite right, because border-box doesn't include margins, where as the block width does. Lucky for us, TH and TD elements lack a margin to speak of.

On the other hand, Firefox have taken their lead from the first rule of the fixed table layout algorithm, and just use the width property as the width according to the box-sizing: content-box model. And why wouldn't they: it's what it says on the tin.

The work around, to get consistent behaviour across both browsers, is to force the column headings to have a box-sizing: border-box.

Since I'm using Compass/SASS, I can create a mixin to include at the top level of any Dojo grid to fix the problem.

@import "compass/css3";

@mixin dojo-grid {

    .dojoGridRowTable > tbody > tr > th,
    .dojoGridRowTable > tbody > tr > td {
        @include box-sizing(border-box);
    }
}


Luckily, IE9 plays along as well. I'm not sure, and I care less about IE8. Google can get regular updates out for Chrome, regardless of the platform. Firefox is doing it's best to follow suit. I'm inclined not to care much at all for IE if the only way for HTML and CSS bugs fixes to be released is with the next major version of the product (or platform it was designed for).

Friday, December 30, 2011

Baby Steps with Dojo DataGrid and JsonRestStore Error Handling

These holidays I decided to teach myself something new.  I decided to take a look at Dojo, for being able to offer professional looking and behaving UI components in web based developments.

Instead of leaping into full prototypes of UI layout, I thought I better keep my first attempt at Dojo relatively simple.  I'm starting with a DataGrid that is using a JsonRestStore as the store.  The JsonRestStore is being serviced by a CakePHP 2.0 backend.  I've written a plugin component that will assist with converting between the JsonRestStore method of paging, and the CakePHP method of paging, but that's a topic for another post, and perhaps a release to GitHub.

For this experiment, I ended up using Dojo 1.6.1, even though Dojo 1.7.1 is available.  When I first started playing with Dojo, 1.7.1 had been released, but it was not available on the CDN.  Then I tried playing with an example that defined a custom module to interact with CakePHP pagination.  Unfortunately, there was a bug in 1.7.0 that meant mixing CDNs and local custom modules was out.  So I downloaded 1.7.1.  Then there was another bug that meant mixing local custom modules with the locale Dojo path was also out.  So I ended up falling back to 1.6.1. After all, all the tutorials were based on 1.6 anyway.  It's doco like this that makes me appreciate the work that the developers at cakephp.org put in to the manual, before the release goes live.  Cynical aside time: I recall watching a Dojo promo for what must have been the 1.6 release.  Many of the featured developers commented how the first step to learning Dojo was to look at the code.  Only a couple mentioned going through the tutorials.  One mentioned jumping on IRC, but was quick to qualify that you should only ask questions if you knew what you were talking about.  Dojos' current documentation might be an improvement on what it used to be, but it falls short of the mark if the recommendation is to look at the code, rather than read the manual.  On the flipside, the featured developers were right.  You're going to have to dive in to the code, given the doco in its current form.  Anyway, on with the show.

My end goal is to have a DataGrid that will allow inline editing, but report errors in saving in a manner that is consistent with CakePHP forms.  That is, after the post, the form keeps the edited values, but displays an indicator and message next to each offending field.  In the case of the grid, I've opted for a red border on the offending cell, and a tooltip that will display the associated error message on mouse over of the cell.

The first thing I did was run up a basic DataGrid using a JsonRestStore.  Inside the dojo.ready(), I defined the data store and the grid.

var dataStore = new dojox.data.JsonRestStore({
    target:"/my_datas/"          
});

var layout = [[
    {'name': 'Column 1', 'field': 'id', 'width': '100px'},
    {'name': 'Column 2', 'field': 'col2', 'width': '100px', cellType:dojox.grid.cells.Bool, styles:'text-align:center;'},
    {'name': 'Column 3', 'field': 'col3', 'width': '200px'},
    {'name': 'Column 4', 'field': 'col4', 'width': '150px'}
]];

var grid = new dojox.grid.DataGrid({
    id: 'grid',
    store: dataStore,
    structure: layout,
    rowSelector: '20px'
}, "gridDiv");

grid.startup();

Using examples found around the usual Dojo haunts, I added Add Row and Remove Selected Rows buttons, and functionality. The layout was updated to make columns 2 to 4 editable.

The Add and Remove functions worked well enough, but the inline editing wasn't persisting the changes to the server.  Here's where I learn my first DataGrid lesson.  DataGrid will sync the data to the store, but if the store needs persisting to a server, like a JsonRestStore does, then you are responsible for doing that yourself.  This should have been evident with the add and remove actions associated with the buttons calling dataStore.save().  So I set up a simple action for the onApplyEdit event to save the dataStore.

var applyEdit = function(rowIdx) {
    dataStore.save();
};
dojo.connect(grid, "onApplyEdit", applyEdit);

I should note that my code didn't actually look like that at the time. I had actually overridden the onApplyEdit directly in the grid definition. Since then, I've come to appreciate that if you're going to provide a function to the constructor arguments, defining one to a variable, then using dojo.connect() is the best course of action, since this will append your function to be called with the event, instead of overriding it. I think. I'm still a Dojo noob, so I'm just going to use dojo.connect() because it seems like the right thing to do.

Well, that editing is great if all is going well.  But if there's an error with the save action, or at least validation is failing, what to do?

Well, I spent hours trying to find an example of error handling for DataGrids and JsonRestStores, but I couldn't find a bloody thing.  Error handling is definitely something that you end up having to write yourself.  Here's how I deal with it.

First up, I needed to communicate the fact that validation had failed for an edit.  With CakePHP, you end up with a validationErrors array in your controller that gets passed to the view, and used in the form.  I created an error element that would display the session flash and the validationErrors in a JSON array.  I would strip the model name out of the validationErrors array, so I didn't have to deal with it in Dojo.  And if I did detect an error in the save, then I would return a 409 status code.

The next part was recording those errors in a place where I could then get at it to change how the grid was rendered.  First things first, storage.  And the dataStore was the best place for that storage.  Here is the modification to the applyEdit function, storing any validation errors.

var applyEdit = function(rowIdx) {

    var actions = dataStore.save();

    dojo.forEach(actions, function(action){
        var result = action.deferred.then(function(){
            if (action.target.validationErrors) {
                delete action.target.validationErrors;
            }
        },function(err){
            if (err.responseText) {                        
                try {
                    var responseText = JSON.parse(err.responseText);
                    action.target.validationErrors = responseText.validationErrors;
                } catch (e) {};
            }
        });
    });          
}; 

I eventually figured out that I would need to use Deferreds to be able to get a hook in to handling errors that might be returned in the JsonRest response for the JsonRestStore. Since it is possible for the save action on a store to save more than just one changed record, I would need to loop through all of the actions that were sent to the service, and perform the error checking against each one.

Thusly, I've attached a then() to the action.deferred. The first function in the then() is run on success.  This one will clear any validation errors for the target row, if any previously existed.  The second function is run if there was an error. Hopefully, the error responseText is parsable JSON containing validation errors.  I guess this part could do with further hardening.  What if the response didn't contain parsable JSON?  What if it did, but didn't contain validation errors?

Error handling in DataGrids with JsonRestStore is such a big deal.  It's such a pity that the subject does not get directly addressed in the available tutorials.

Now that I have the validation errors in the dataStore, I can use a cell formatter to detect and highlight the offending cell with a red border.  I'm actually just going to apply a CSS class, and let CSS take care of the rest.

var formatter = function(val, rowIdx, cell) {
    var item = grid.getItem(rowIdx);

    if (item.validationErrors && item.validationErrors[cell.field]) {
        cell.customClasses.push("validation-error");
    }
    return val;
};

Also add formatter: formatter to the records in the layout variable that are editable, namely, columns 2, 3 and 4. You'll also need to define the CSS.  You'll need to be specific, if you hope to have your CSS get included ahead of the other styles defined.

.claro .dojoxGridCell.validation-error {
    border: 1px solid red;
}

While I was at it, I also added the code for the tooltip that would display on mouse over of the offending cell.  This part of my code, I do believe is flawed, but I'll chat about that after the code.  First well, need some code to show the tooltip on mouse over, and hide it on mouse out.  I've also tried to add some code to prevent the tooltip from showing while the field is being edited.  I do this because I believe the HTML node of  the cell is actually replaced during editing, the lost with it is the reference to the displayed tooltip.

var tooltipNode;
var showTooltip = function(e) {
    if (gridTooltipEnabled) {
 var msg;
 var item = e.grid.getItem(e.rowIndex);
 if (item.validationErrors && item.validationErrors[e.cell.field]) {
     msg = item.validationErrors[e.cell.field].join('
');
 }
 if (msg) {
     dijit.showTooltip(msg, e.cellNode);
     // hold reference to cellNode with tooltip for easy hiding
     tooltipNode = e.cellNode;
 }
    }
};

var hideTooltip = function(e) {
    dijit.hideTooltip(e.cellNode);
    tooltipNode = null;
};

var gridTooltipEnabled = true;
// grid definition here ...

dojo.connect(grid, "onCellMouseOver", showTooltip);
dojo.connect(grid, "onCellMouseOut", hideTooltip);
  
// disable and hide tooltip while editing
dojo.connect(grid, "onStartEdit", function (cell, rowIdx) {
    gridTooltipEnabled = false;
    dijit.hideTooltip(tooltipNode); // can't get to a cellNode from cell
});

// enable tooltip when finished editing
dojo.connect(grid, "onCancelEdit", function(rowIdx) {
    gridTooltipEnabled = true;
});

This code constructs a tooltip from the validation errors, and displays it. If you start editing a cell, it will attempt to hide the tooltip, and prevent another from being displayed until you have cancelled editing or have applied the edit (I have a gridTooltipEnabled = true; at the top of applyEdit()).

What I really wanted to was to be able to identify a cellNode from the cell argument passed to onStartEdit, and call dijit.hideTooltip() against that.  But I just couldn't find a way to link these two items together.

The flaw is that if two tooltips are on display when editing starts, then only one of them, the most recently activated, is going to be hidden.

The last part to go is the behaviour of retaining the entered data after an error, and only removing it if the edit is cancelled, or another cell in the store is successfully saved.  You can't micro manage a JsonDataStore.  When you call save(), it applies for all dirty objects in the store, and when you call revert(), it applies for all dirty objects in the store.

To achieve this behaviour, you need to use the revertOnError property as an argument to the dataStore.save() function in applyEdit().  A slight side effect to this property is that the cell will not get redrawn if there is an error.  As a result, formatter is not called, and you don't get to see the red border until you go to a different page of data in the grid, and then go back.  To remedy this, you need to force a render with a grid.update() after assigning validationErrors to the target row in applyEdit().

However, if the user wishes to not attempt the fix the validation errors, and wishes to cancel the whole change, they can start editing the cell, and then cancel edit by pressing Escape.  I've added some code to the onCancelEdit action to revert if the row was dirty, and had validation errors.

dojo.connect(grid, "onCancelEdit", function(rowIdx) {
    gridTooltipEnabled = true;

    // if canceling editing and previously had errors, clear the errors and revert
    var item = grid.getItem(rowIdx);
    if (item) {
        if (dataStore.isDirty(item) && item.validationErrors) {
            // would rather revert the individual item, but this will do
            dataStore.revert();
        }
    }
});

As mentioned in the comments, I would rather just revert the individual row, instead of reverting the whole store. To do this, I have a feeling I would need to perform an explicit fetch on the item, and replace the dirty item in the store. I'll keep it simple for the moment.

There was a Dojo bug I had to apply a workaround for, when using the revertOnError property.  JsonRest.commit() has a bad reference to a dirtyObject variable that doesn't exist.  Simply changing it to dirtyObjects doesn't work either, but the bug reporter was good enough to provide a patch that did work when the problem was reported almost two years ago.  Unfortunately, this means that I'll never be able to use this with a CDN, and I'd need to patch and rebuild the Dojo 1.6.1 distribution.  My next task will be to upgrade this to use Dojo 1.7.1 (or 1.7.2, if it has been released by then).  Hopefully this bug will be fixed by then, or perhaps I'll be able to bump the ticket.

Well, that was my first look at Dojo, and in particular, the DataGrid and JsonRestStore.  I'm hoping that all the tutorials get an upgrade, the documentation improves and show stoppers like no local custom modules with CDNs are fixed before too long.  Even though this was just baby steps, it still seemed like I had to delve to an intermediate level to get the functionality to a point where it would be useful.

If  I can get this working with the 1.7.1 CDN, then I'll load the example to GitHub.  I'll also do an article on the CakePHP plugin that assists with JsonRestStore paging.