Tag Archive for 'AJAX'

Monads and Asynchronous Javascript

Published
by
Victor Nicollet
on May 11, 2011
in Functional
. 3 Comments

A minor yet interesting syntax extension for Javascript would help solve the eternal problem of too many nested anonymous functions when writing asynchronous code. For instance, reacting to an AJAX request in standard jQuery looks like this:

$.getJSON('/status.php', {id : the_id}, function(data) {
  if (data.deleted) {
    $('#' + the_id).hide(500, function(){
      $(this).remove();
    });
  }
}

The function nesting is ugly, but necessary. In a monadic writing style, it would look like this instead :

var! data = $.getJSON('/status.php', {id : the_id});
if (data.deleted) {
  do! $('#' + the_id).hide(500);
  $(this).remove();
}

This style restores the imperative look of the function, hiding away the asynchronous nature of the code in the special keywords.

In itself, the rewriting is pretty simple to perform based on two unambiguous rules :

var! a, b, c, d = expr(x,y,z);
more code

Becomes :

expr(x,y,z,function(a,b,c,d) {
  more code
});

And there is a shorthand notation:

do! expr(x,y,z);
more code

That becomes :

expr(x,y,z,function(){
  more code
});

As a final example, here is how the “Rate Me” jQuery example would be written :

do! $(document).ready();

// generate markup
$("#rating").append("Please rate: ");

for ( var i = 1; i <= 5; i++ )
  $("#rating").append("<a href='#'>" + i + "</a> ");

{
  // add markup to container and apply click handlers to anchors
  var! e = $("#rating a").click();

  // stop normal link click
  e.preventDefault();

  // send request
  var! xml = $.post("rate.php", {rating: $(this).html()});

  // format and output result
  $("#rating").html(
    "Thanks for rating, current average: " +
    $("average", xml).text() +
    ", number of votes: " +
    $("count", xml).text()
  );
}

Doesn’t that look nicer ? I certainly look forward to something similar being included in ECMAScript.

Last Minute Skin

Published
by
Victor Nicollet
on September 4, 2009
in Functional
. 2 Comments

Right now, we render our page layout on the server, thus wasting precious bandwidth sending the same header, footer and menus all over again every single time. AJAX techniques have evolved to reload only the inner part of every page, but they require clever URL manipulation or ‘back’, ‘refresh’ and bookmarks won’t work, and they impose strong constraints on page layout and on the way the server responds to requests.

Why not do it the other way around? Have every page include the same layout-generating JavaScript file (kept in the browser cache for optimum performance) ! This is the idea behind the last-minute-skin pattern.

AJAX is Hard

Published
by
Victor Nicollet
on August 31, 2009
in Imperative
. 0 Comments

Seen from the outside, AJAX has become an easy technology:

$('#container').load('http://domain/path/to/page');

Even if you’re doing smarter things, like updating server-side values with asynchronous POST requests, it’s still easy:

$.post(
  'http://domain/path/to/action',
  { user_id       : $('#user').val(),
    new_user_name : $('#name').val() }
);

And, of course, it’s also easy to make mistakes in AJAX.

Not taking errors into account

In an ideal world, the AJAX request is sent to the server, completed successfully, and the response is propagated back and applied.

In the real world, the AJAX request might never reach the server because the network cable was pulled, or it could carry stale data that cannot be processed, or the user session might have expired, or something else altogether.

This “request cannot be completed successfully” issue has been solved for years in the traditional HTTP world by both servers and browsers: when you try to get to a page and that page can’t be reached, you will either get an error message from your browser or be redirected to another page by the server.

In the AJAX world, a failed request times out silently without anything happening. You have to actually implement that small “Your session has expired, click here to log in again” message box yourself, just like so many other websites did. And, of course, you need to take into account into all of your workflows that the user may be logged out of their session at any point.

Don’t forget to include cable-plugging as part of your testing protocols!

Forgetting to refresh parts

When you post some modifications to the server asynchronously, you need to refresh some parts according to the new state of the server. Which parts do you refresh?

While the answer might seen easy in every single specific case (I’m updating this list/object/grid, so I’ll just refresh it), the general answer is not so simple: your server-side modification might have an impact on other parts of your system.

Consider a typical Facebook-like interface: you have a menu with an inbox, and to the right of that inbox there’s the number of unread messages. On the inbox page, you have a list of messages with a little cross on each message that deletes it through AJAX. The naive thing to do is have that cross update the list of messages, but then deleting an unread message wouldn’t update the menu.

Inevitably, a developer working on an AJAX feature will forget to take into account that some other part of the page that needs to be updated. Or a developer will add some information to every page and forget that some pages need to update that information.

Repeating yourself

Javascript does not benefit from the same clean separation of features into classes, files, packages and namespaces. Also, IDE quality is lacking when compared to other languages. This makes it hard to refactor JS code when duplicate functionality starts to appear.

Let’s consider the asynchronous post situation. In order for that code to work, you need to have fields with identifiers ‘user’ and ‘name’ and some element to initiate the post through an event. This is not encapsulated: if another page needs similar “post user name” functionality, the code will have to be rewritten. In fact, when the code is that small, it’s actually faster to rewrite it than it is to find and call an existing function (not to mention writing that function in the first place).

No refactoring means the code repeats itself. Having two user-name-change pieces of javascript on two different pages means twice as much work to do when you eventually change how that part really works.

Allowing complex behavior to be written in two or three lines is no excuse for letting your code get out of hand: stand firm by the “once, twice, refactor” motto and do not hesitate to turn a three-liner into a ten-line reusable function with appropriate documentation.

JSOS : JS-PHP mapping

Published
by
Victor Nicollet
on August 7, 2009
in Dynamic
. 0 Comments

I’ve been working on Javascript-PHP remote call mapping techniques. A set of PHP classes with static functions are selected as a public interface and automatically exported so that the Javascript can call them. Usually, this kind of mapping involves three difficult points:

  • Detecting what functions are exported and building the appropriate JS code.
  • Detecting what function the JS is calling.
  • Transforming data between JS and PHP.

I solve these with glob(), __autoload and json_encode (respectively) in a little prototype I called JSOS (JavaScript Or Something). jQuery on the client side provides me with synchronous AJAX queries. The code (PHP with echoed Javascript) looks like this, and is placed in a controller:

<?php
 function __autoload($classname)
 {
   require_once($classname . '.inc.php');
 }

 if (!isset($_POST['cls'])) {
   echo '<html><head><title>JSOS</title>';
   echo '<script type="text/javascript" src="jquery.js"></script>';
   echo '<script type="text/javascript">var jsos={};';
   echo '$.ajaxSetup({async:false,timeout:5000});';
   echo 'jsos.$=function(c,f,a){';
   echo 'var o={exception:"Server disconnect"},';
   echo 't={cls:c,func:f};';
   echo 'for(var i in a)t[""+i]=a[i];$.post(".",t,';
   echo 'function(d){o=d},"json");if("exception" in o)throw o.exception;';
   echo 'return o.result};';

   foreach (glob('*.inc.php') as $file)
   {
     $class = str_replace('.inc.php', '', $file);
     echo 'jsos.' . strtolower($class) . '={};';
     $text = file_get_contents($file);

     preg_match_all('/public static function ([A-Za-z_0-9]+)\(([^)]*)\)/',
                    $text, $func);

     foreach ($func[1] as $id => $value) {
       echo 'jsos.'.strtolower($class).'.'.strtolower($value).'=function(';

       $args = explode(',', $func[2][$id]);

       foreach (array_keys($args) as $key) {
         $args[$key] = str_replace(array(' ', '$'), '', $args[$key]);
       }

       $args = array_filter($args);

       echo implode(',', $args).'){return jsos.$';
       echo '("'.$class.'","'.strtolower($value).'",['.implode(',',$args).'])};';
     }
   }

   echo '</script></head><body></body></html>';
   exit;
 }

 $args = array();
 foreach ($_POST as $key => $val) {
   if ($key === 'cls')
     $cls = $val;
   elseif ($key === 'func')
     $func = $val;
   else
     $args[(int)$key] = $val;
 }

 ksort($args);

 if (!class_exists($cls)) {
   echo '{exception:"No such package!"}';
   exit;
 }

 if (!method_exists($cls, $func)) {
   echo '{exception:"No such method!"}';
   exit;
 }

 try {
   $result = call_user_func_array(array($cls, $func), $args);
   echo json_encode(compact('result'));
   exit;
 }
 catch (Exception $e) {
   $exception = "$e";
   echo json_encode(compact('exception'));
   exit;
 }

All files in the current directory with a ‘.inc.php’ extension are assumed to contain a similarly-named class, and all public static functions of that class are exported. For example, suppose Test.inc.php contains the following definition:

class Test
{
  public static function Run($a) { return "$a$a"; }
}

Then, to call the Test::Run function from the above page, one would simply type in Javascript:

jsos.test.run('Hello');

And indeed, with only five lines of code, the result to a client request is computed on the server and displayed on the client again, without issues. Here’s the result running in Firebug:

jsos

Further work should include handling errors (right now, if the server encounters an error or outputs non-JSON data, the call will die with a “Server disconnect” exception), which makes debugging easier than having to wade through the Net tab of Firebug.

Easier Unit Tests

Published
by
Victor Nicollet
on August 6, 2009
in Dynamic
. 2 Comments

Automated unit testing has three main advantages:

  1. It forces you to express in detail what you want the unit to do (so that a computer may then test it by checking the results).
  2. For an unit to be testable, its results should be checked as “correct” or “incorrect” on their own or with minimal context. This makes code easier to reuse.
  3. Since it’s automatic, you do not need to test manually every time you make a change: the tests will run hourly or nightly (or for every build) and tell you what went wrong.

The cost, however, is that unit testing code must be written for every unit. That code needs to create an object, manipulate it, then check its return values for validity. And test-driven-development advocates insist that such code should be written before the actual unit (make it compile, make it fail, make it pass).

I do understand their position: if you’re thinking of very specific corner cases that you might forget about while writing the unit, you might as well write tests for these corner cases first. But what about the simple, core functionality of the units? If a programmer sets their minds to testing an unit, they can usually look at the value and decide “it’s correct” or “it’s wrong”, and that process is orders of magnitude faster than writing an assertion that checks whether the value is correct. We’re not talking about complex ten-argument dozen-property function on a deep-inherited object here, it’s more of a “this string goes in, that string goes out” concept.

An example would be a “slugify” function : plug an arbitrary string into it, and a cleaned up lowercase hyphen-separated string comes out. As long as you are mindful of corner cases (weird characters, encodings, empty strings and so on) testing this function manually is quite simple. You certainly lose the benefit of point 1 (no more expressing the details of what the function should do in advance of writing it) but you can keep the benefits of 2 (whether automatic or manual, tests make code reusable) and you can even get the benefits of 3 by saving your code as an unit test.

This is what happens in the console-like scaffold I have been working on (you can click to enlarge):

autotest

You type in code in the text box. Clicking “Run” (or pressing TAB RET) sends the code through AJAX to the server and retrieves the var_dumped result (as well as a highlighted version of the code). The mindful programmer can use this console to preemptively test and debug code. The excellent programmer will even adapt their code to make such testing easier, and being able to use any kind of code in the barren context of this console means that code can be used anywhere.

The real hit there is that little Add Cog icon from the FamFamFam Silk icon set. It’s truly beautiful and adds a touch of antialiased pixel art to an otherwise bland rounded-edges Web 2.0 console. And what it does is even better: when clicking that icon, the corresponding piece of PHP is automatically added to the database of unit tests, as a test that runs the code on the green background and asserts that the output of the code is identical to the text on the grey background.

This makes the “for every bug you find, write an unit test that finds that bug” insanely easy to do: find the bug using the console, correct the code, check with the console that the bug is dealt with, and then add the final “bug corrected” console run to the unit test database. The work spent debugging a program from this console is never lost, it all ends up being an automated test with one single click.

This also makes writing unit tests much more developer-friendly: the console lets you test your code as you write it, without having to write controllers or views or whatever other contraption you need to display the results. An excellent choice for testing-averse developers.

HTTP-friendliness

Published
by
Victor Nicollet
on February 17, 2009
in Dynamic
. 0 Comments

When I use a website, I expect it to support the basic ideas of the HTTP protocol. Otherwise, I will not be able to use my browser to the full extend of its “back”, “forward”, “refresh” and “bookmark” capabilities. This means:

  • GET requests return data without any important side-effects, so that I can refresh or bookmark that page and still get the same results without any risk of repeating an operation or getting some entirely different piece of data.

A GET request is named GET for a reason. If it was meant to post data to a website, it would have been called POST. No, wait, there’s also a POST method for that. Once you see a GET request as a read-only access to a named resource (with the URI being the identifier) you get the full power of HTTP support from browsers. When I visit a certain URL as GET, I expect it to show me the resource corresponding to that URL or a newer version of that resource or a redirection to a page explaining why I cannot access it and how I can (if I can) get to access it.

  • POST requests return a page that’s a summary of what they did, and should make it clear that refreshing the page will repeat that operation.

No displaying of a GET-like page that gives the illusion of being a read-only request. In fact, don’t even give your users the impression that a POST request is somehow intended for reading data from your server. A POST should be dedicated entirely to posting data, with the courtesy response intended to provide concise information about how it went and what to do next. If you don’t have anything interesting to display that is directly related to the operation you just did, redirect to a GET.

  • HTTP has no status. Sure, you can get some status by using cookies (client-side) combined with sessions (server-side). But keep in mind that an user may have several windows opened on your website, and there may be long durations (or even computer changes) in-between visits.

A very common (and annoying) thing to do is use sessions to add status where status should not be present. Especially if you have AJAX to take care of most persistence needs you could come up with in a client-side fashion. What I usually accept as status is authentication information (because this is the elementary point of having a session in the first place) and information about what the user has recently done (such as displaying back error messages on a form, or a success message, on the GET page you end up on after visiting a POST). The good news is that most develpers have been bitten often enough to avoid placing important information in the session, precisely because they’ve been hit hard when the session went away.

My ideal website provides me with a list of resources I can read, and each resource is identified by an URL that I can send a GET to. A session mechanism may alter what I see when I visit a resource, for instance if I’m not authenticated or if I don’t have the necessary permissions. Then, if I am expected to interact with the contents of the website, I am provided with POST URLs that redirect me to a GET URL that makes sense. A login form would be a nice place for a POST (since it registers a session) while a search form would be a nice place for a GET (since a search page is a read-only resource that needs no server-side modifications). If I need some smart state-based behavior to occur, it should be provided to me as AJAX, with the implication that it won’t persist if I leave the page (the “you cannot save web pages” principle has been fairly well assimilated by users). Informational state, such as error messages, can be provided to pages by my session data.

AJAX Without Javascript

Published
by
Victor Nicollet
on September 24, 2008
in Dynamic
. 1 Comment

AJAX is an acronym for Asynchronous Javascript And XML. It allows changing parts of a page based on data present on the server without having to reload the entire page.

The technical solution involves some javascript code that is executed by the browser and sends an HTTP request to the server. The server then responds with the appropriate data and the javascript code parses the data (originally in XML, now currently in the JSON serialized format) and displays it where required.

Writing AJAX by hand quickly became old, and instead several javascript libraries provide facilities to perform AJAX queries. One such library (used in this example) is jQuery.

The problem

AJAX still requires a reasonable amount of work both on the server and on the client. Server-side, the programmer has to set an access path so that the AJAX requests are received and handled, and must also handle the sending back of data to the page. Client-side, the programmer still has to specify what data to put where.

Let’s consider a simple problem: a page has two buttons labeled “Increment” and “Reset”, a numerical counter (which is incremented by the increment button and reset by the reset button), and a short sentence which describes how many times the counter has been reset.

Ideally, the page must satisfy that:

  • Clicking the Increment button may only have to reload the two counters (the actual counter, and the increment count).
  • Clicking the Reset button may only have to reload the actual counter, without touching the increment count.
  • There should not be two code paths based on whether the page is being loaded through normal means or altered by clicks.
  • Every button click must save the data on the server so that it sticks around on the next visit or when reloading.

The solution

In a clean MVC fashion, the Model is the $_SESSION variable provided by PHP. The controller is a set of functions that alter and print the model:

session_start();
class Controller
{
  function url()   { return 'http://www.example.com/mypage.php'; }
  function incr()  { $_SESSION['value'] ++; $_SESSION['incr'] ++;}
  function reset() { $_SESSION['value'] = 0; }
  function value() { return (string)$_SESSION['value']; }
  function incrs() { return (string)$_SESSION['incr']; }
}

And the view is a special script that is to be compiled by a special compiler:

<html>
  <head>
<?
$this -> action('js_incr' , 'incr',  array('value', 'incrs'))
      -> action('js_reset', 'reset', array('value'));
?>
    <title>Example page</title>
  </head>
  <body>
<ul>
  <li>
    <button onclick="js_incr({})">Increment</button>
    <button onclick="js_reset({})">Reset</button>
  </li>
  <li>
    The value is <? $this -> span('value'); ?>.
  </li>
  <li>
    The value has been incremented <? $this -> span('incrs'); ?> times.
  </li>
</ul>
  </body>
</html>

The compiler itself is run as follows (of course, it’s also possible to save the compiled code to a file and load it without having to compile it again):

require_once( 'AjaxCompiler.php' );
$compiler = new AjaxCompiler('view.php');
$ajax = $compiler -> getCompiledObject(); 

print $ajax -> react($_POST, new Controller());

The compiler executes the file, keeping everything in HTML format but replacing the function calls with the appropriate data. The action function specifies that a javascript function (named after the first argument) must send all its arguments to a PHP function on the server (named after the second argument) that is a member function of the controller (the controller is the second argument to the $ajax -> react() call) and return that function’s output converted to javascript, then refresh the spans and divs named in its third argument.

Spans and divs are generated with the appropriately named functions of the controller (which may use any means of rendering seen fit) and sent along with the AJAX response to the original query so that they are replaced in the XHTML document. All the binding is done automatically by the compiler, so that the same functions are used for generation both in normal querying mode and in AJAX mode, and yield identical results for normal XHTML.

Work in progress

Many things remain to be done. hierarchical AJAX documents are still not possible, nor are parametric spans and divs (for instance, for editing individual elements of a list of elements). Also, the system has not yet been tested with javascript-enabled data in those spans and divs that are se

1170 feed subscribers
(readers who polled a feed this week)