Tag Archive for 'Bugs'

Lorem Ipsum

Published
by
Victor Nicollet
on March 3, 2010
in Imperative
. 1 Comment

Lorem Ipsum is a sample phrase used as a filler in typesetting, to illustrate how some text would look. Here’s a sample paragraph:

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

This approach to filler text is superior in many ways to most alternatives:

  • It’s long, certainly longer than “Test” or “AAA”, so it can fill several lines (and test whether there is an unexpected length limit when saving the data).
  • Unlike random strings of characters copy-pasted several times, it is split into words of uneven length, spaces between words do not align horizontally or vertically.
  • It is readily recognizable as random text by any typesetter (or developer) worth its salt.

A typical testing strategy, when filling forms by hand, is to copy-paste one or two Lorem Ipsum paragraph to test such things as how the text area reacts, whether it is saved correctly, and so on.

Lorem Ipsum does have some limitations:

  • It’s written in latin, so it fits nicely in the ASCII range of characters. As such, it does not test for Unicode support.
  • It contains no quotes of any kind, so no testing of database escaping processing either.
  • It contains no HTML-specific characters like < or &, so HTML character escaping is not tested either.
  • For that matter, it does not contain exceedingly long words that would overflow a single line, so you cannot test for this kind of overflow either.
  • Sometimes, you want to auto-linkify links and URLs.
  • Sometimes, Skype turns numbers into … clickable numbers.

I need to test these things on my web applications, so I’ve developed my own version of a “Modern Lorem Ipsum“:

Lorem <a href=”javascript:document.write(”)”>ipsum</a> dòlor sit àmet, consectetur adipisicing élit, sèd do eiusmod tempor incididunt ut labore & dolore magna aliqua. <hr/>Ut enim@minim.com veniam, quis nostrud exercitation `”ullamco laboris nisi & aliquip ex æ commodo consequat. Duis aute irure dolor 01 23 45 67 89 in reprehenderit in voluptate velit esse cillum dolore `eu fugiat https://nulla.biz/pariatur. aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Excepteur [u]sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit ‘anim id est laborum.[u] Lorem” ipsum dòlor sit àmet, consectetur adipisicing élit, sèd do eiusmod tempor http://incididunt.ut.com/labore & dolore magna aliqua. <b>Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex æ commodo consequat.</b> “Duis aute irure dolor in ‘reprehenderit in voluptate velit — esse cillum dolore eu fugiat nulla pariatur.

Feel free to copy-paste it away. Wordpress certainly does seem to have a hard time with these long lines in post bodies—I wonder if it happens in comments as well :)

That DOM removal thing, again

Published
by
Victor Nicollet
on January 22, 2010
in Dynamic
. 0 Comments

Earlier this month, I pondered what looked like a bug in JavaScript/DOM/jQuery: removing an element from the DOM with jQuery (either manually with remove() or by setting the html() of its parent to something else) kept most of the data bound to the element around, but removed all event handlers from it. You could then re-insert the element, but its event handlers would be lost.

I then gathered from several sources, such as Stack Overflow, that this is a jQuery issue (or rather, feature) and not a JavaScript one.

The underlying cause is explained by Douglas Crockford:

When a DOM object contains a reference to a JavaScript object (such an event handling function), and when that JavaScript object contains a reference to that DOM object, then a cyclic structure is formed. This is not in itself a problem. At such time as there are no other references to the DOM object and the event handler, then the garbage collector (an automatic memory resource manager) will reclaim them both, allowing their space to be reallocated. The JavaScript garbage collector understands about cycles and is not confused by them. Unfortunately, IE’s DOM is not managed by JScript. It has its own memory manager that does not understand about cycles and so gets very confused. As a result, when cycles occur, memory reclamation does not occur.

A common solution to this problem is to remove the cycles when the element is removed from the DOM. Since a major source of cycles in your average jQuery program is the presence of event handlers, then removing the event handlers when an element is removed from the DOM solves the problem most of the time.

With the release of jQuery 1.4, the new documentation for .remove() makes mention of this fact:

In addition to the elements themselves, all bound events and jQuery data associated with the elements are removed.

The documentation for .html() still makes no mention of this. If you want to remove an element and keep all the goodies you bound to it, jQuery 1.4 provides you with .detach():

The .detach() method is the same as .remove(), except that .detach() keeps all jQuery data associated with the removed elements. This method is useful when removed elements are to be reinserted into the DOM at a later time.

Shouldn’t Happen…

Published
by
Victor Nicollet
on January 18, 2010
in Imperative
. 0 Comments

Design and development is turning the great unknown chaos into tiny bits of controlled functionality with promises about what the result will be, and expectations about what the input should be.

There is an interesting duality between two categories of expectations, depending on whether they are the responsibility of the user, or of the programmer.

User errors are classic mistakes involving incorrect input, such as attempting to load a file that does not have the right format, or visiting a web site that does not exist, or entering an incorrect email address. A program is expected to, at the very least, gracefully handle these situations (because nobody likes errors) and the best programs are actively designed to reduce the possibility of error though appropriate user interface choices.

Programmer errors are the most frequent ones, but most of there are luckily caught by a compiler (or, in the case of the less lucky interpreted languages, the parser). The basic idea is that if you expect a function parameter to be an integer, and you tell your compiler, then static analysis will determine that you will receive a string argument, and the universe will collapse build will fail.

Static Analysis

Static analysis can be very smart. It can prove beyond any doubts complex properties about complex software written in obscenely low-level code (such as C with inline assembly). The problem if that working with a static analysis tool can add unusual constraints on the developers themselves: the halting problem dictates that no tool can safely predict the behavior of a program, so any given tool will either have false negatives (undetected bugs) or false positives (safe code reported as dangerous) and the general trend for static analysis tools is to avoid any false negatives at the cost of false positives.

The quality of a static analysis tool is determined by how hard it is to write code without false positives (usually done by manually coding around the blind spots of the tool).

Static analysis tools have two problems. One, they’re not available for every single language and platform out there. Some of use are still using languages with eval(), throwing Java exception-safety out the window because we find it too constraining, doing without those pesky type systems and generally making a childish fuss about those “warning” thingies. Two, static analysis tools can only check constraints that are described by the developer in some form, such as assertions, preconditions, postconditions, type annotations or some other kind of attribute added to the code.

So, if you forget to “assert” it, nobody is going to check it for you. For instance, no tool is going to warn you that you unwittingly leak a credit card number to a third party.

The Elephant Statue

In a sense, predicting user errors is the mirror activity of gathering specifications. Both force you to think about all possible situations your software will face, and decide what should happen: maybe you have to display an error, maybe you will have to tread the input in a clever but predictable way, or maybe you will have to rework your process to prevent that situation from happening.

This is akin to creating an elephant statue by starting with a block of stone and carving out everything but the elephant. Deciding what your users can do implicitly defines what your users cannot do. Depending on the situation, you may guide your design with either approach.

DOM removal and events

Published
by
Victor Nicollet
on January 7, 2010
in Functional
. 4 Comments

Let’s try something… go to a page with jQuery enabled (such as this one), and run the following code in your Javascript debugger console (such as Firebug):

var button =
  $('<button>Click me</button>')
  .click(function(){alert('Clicked!')})
  .appendTo('body')

In case you were wondering, this creates a brand new button, causes it to display a “Clicked!” message box when it’s clicked, and appends it to the document you are viewing.

Click on the button that just appeared : the message box appears. Not very surprising.

Now, run the following code on the same page :

$('body').html('');
button.appendTo('body')

As expected, everything on the page, including the button, disappears. However, the button is still referenced by the button variable, so it sticks around and we can append it back to the document. And indeed, it does appear on the page.

Click on the button again. This time, no message box appears.

I honestly have no idea why.

December 2009 PDF Vulnerability

Published
by
Victor Nicollet
on January 5, 2010
in Functional
. 0 Comments

All file formats follow the same evolution.

  1. They start by grouping together some static content, with some nifty features for presenting and editing that data. Think text files, bitmaps, RTF documents… The file format is reasonably easy to understand, and the reader/writer is so simple that it would take a bad programmer to create vulnerabilities.
  2. Then, they start including plug-ins that let them handle more and more types of contents. This lets you include an image inside an HTML page or an Excel spreadsheet in a Word document. This relies on many plugins for getting things right. It sometimes happens that a given plugin contains a security fault that can then be exploited, for instance Internet Explorer had an issue with images in PNG format. The user would visit a page, that page would display an image, and the computer would be contaminated.
  3. Finally, they need to become interactive, so they include a scripting language of some sort. Excel has a macro system that uses Visual Basic, HTML includes Javascript…

The PDF format followed the same process to end up where it is now. In addition to any static document data (text, vector and raster images) and extended content (flash animations, videos, reader extension signatures) a PDF also contains short JavaScript that let authors create interactive documents. This means a PDF document on your desktop can:

  • Accept user input (such as checkboxes or text fields). The input can be saved to the disk if the reader supports it and allows it (Acrobat Reader, used by the vast majority of computer users, only allows saving a file if its author purchased a reader extensions license and signed the PDF file with it).
  • Change its layout at will, for instance displaying a “spouse” page only if the “spouse” checkbox was ticked.
  • Be cryptographically signed, and display information about who signed it. This kind of signature is actually accepted as valid legal proof in many countries.
  • Compute a scannable bar code from user input, so that it can be printed, then scanned on the other side with reduced error rates.
  • Send data over the internet. It can even send itself as an attachment to an email.

Needless to say, with all these features, there are inevitably going to be some exploitable security issues in the mix. Being a popular program, like Acrobat Reader, only increases the number of black hat hackers looking for vulnerabilities. One of these is the recent CVE-2009-4324 from December 2009. There are many types of vulnerabilities, their common feature being that they end up executing arbitrary operations on the computer (as opposed to the safe operations Acrobat Reader normally allows). These operations are usually to download or install trojans, so that the attacker can gain complete control over the computer.

CVE-2009-4324 is of the use-after-free kind. In short:

  • it creates a resource (which uses some memory),
  • it frees (destroys) the resource to recycle its memory,
  • it writes something to that memory,
  • it attempts to use the resource

Normally, the program should stop at step four and say “you can’t use the resource, it’s been destroyed”. A bug can cause it to believe that the resource is still there. The programmer probably assumed that the memory still contained a valid resource and did defend against the memory containing something else… and accessing that as if it were a valid resource executes some code that the attacker wanted to execute. Bingo.

In the case of CVE-2009-4324, this happens as part of the Doc.media.newPlayer method which, for performance reasons, was not completely implemented in Javascript—a bug in some Javascript code can cause the document to misbehave, but it cannot do anything that the Javascript couldn’t do on its own. Those parts that were written in a lower-level language, with access to the computer, contained the exploited bug.

The bug causes the processor to start executing code at a different memory location. In an ideal hacker world, that location would be precisely where some nasty code is present. Buffer overflows, when used to rewrite pieces of the stack, do allow such deterministic jumps. However, CVE-2009-4324 only allows a jump to an undetermined location.

The hacker solution is to use heap spray. The basic idea is that you have a short piece of code you want to execute (the payload). You create a block from that payload by adding no-ops (machine instructions that say “skip me”) before the payload. Then, you create lots of these blocks in memory, and trigger the exploit.

The exploit causes the computer to jump to an undetermined memory location. If it falls within the no-op section of any of the blocks you’ve created, you win: the computer skips over the no-ops, reaches the payload and executes it. If not, the program will crash. Too bad…

Can references be null?

Published
by
Victor Nicollet
on November 3, 2009
in Imperative
. 7 Comments

I often hear people insist that C++ references can be null.

You can easily create a null reference like this:

int *ptr = 0;
int &ref = *ptr;
// ref is a null reference

First, there is no such thing as a null reference in standard C++.

Like all programming languages, C++ is purely a construction of the human mind, and the concepts of the language have been given arbitrary but meaningful names as part of the standardization process so that the users of the language could understand each other. These could be existing words such as class, aligned, statement, pointer or new ones like rvalue or cv-qualified.

In particular, the C++ standard does define a null pointer, but it does not define a null reference because that concept is not part of the language.

This means that instead of having a standard definition (like a null pointer), null references have a commonly accepted definition that follows a similar schema: a null reference is a reference r such that &r is a null pointer. For the rest of this article, I will follow this definition.

Second, a null reference cannot exist in a well-behaved program.

Since the concept does not appear in the standard, it’s pretty obvious that no place in the standard explicitly mentions an operation resulting in a null reference being constructed. In fact, the description of how references are constructed explains that references are constructed from other values, and by definition values cannot have a null pointer for an address.

In particular, the typical construction of a null reference by dereferencing a null pointer is explicitly mentioned in the standard, and described as undefined behavior.

In short, while many modern implementations will let you create a null reference, this will necessarily involve some form of undefined behavior along the way.

I tend to evaluate programmers on a “would I hire this person?” basis. So, what does the “C++ references can be null” statement tell me about someone ?

  • They might be confusing C++ references with C# or Java references, which can be null. Certainly a NO HIRE for a senior C++ programmer.
  • They might have misread their textbook and genuinely think references can be null in the same way as pointers (except, well, the lack of syntactic sugar for checking if a reference is null or creating a null reference strikes me as odd if that were the case). Again, NO HIRE for a senior C++ programmer.
  • They know it’s not standard, but they do not care about writing standard programs. This is an immediate NO HIRE even for junior C++ programmer.

Then, there are those who know about it, and would not use it because they care about writing standard programs. The convention when discussing C++ is to assume that the program is well-behaved, so most people would parse “C++ references can be null” as “C++ references can be null in a well-behaved program”, the latter being obviously incorrect. I don’t really mind a small misunderstanding, the subject being as complex as it is, as long as it’s corrected quickly.

A person who does not is either someone without much experience discussing C++ with others, or a troll. A definite NO HIRE on a team.

Now we know why…

Published
by
Victor Nicollet
on September 3, 2009
in Random Randomness
. 0 Comments

…the stock market crashed.

failcac

AJAX is Hard

Published
by
Victor Nicollet
on August 31, 2009
in Imperative
. 0 Comments

Seen from the outside, AJAX has become an easy technology:

$('#container').load('http://domain/path/to/page');

Even if you’re doing smarter things, like updating server-side values with asynchronous POST requests, it’s still easy:

$.post(
  'http://domain/path/to/action',
  { user_id       : $('#user').val(),
    new_user_name : $('#name').val() }
);

And, of course, it’s also easy to make mistakes in AJAX.

Not taking errors into account

In an ideal world, the AJAX request is sent to the server, completed successfully, and the response is propagated back and applied.

In the real world, the AJAX request might never reach the server because the network cable was pulled, or it could carry stale data that cannot be processed, or the user session might have expired, or something else altogether.

This “request cannot be completed successfully” issue has been solved for years in the traditional HTTP world by both servers and browsers: when you try to get to a page and that page can’t be reached, you will either get an error message from your browser or be redirected to another page by the server.

In the AJAX world, a failed request times out silently without anything happening. You have to actually implement that small “Your session has expired, click here to log in again” message box yourself, just like so many other websites did. And, of course, you need to take into account into all of your workflows that the user may be logged out of their session at any point.

Don’t forget to include cable-plugging as part of your testing protocols!

Forgetting to refresh parts

When you post some modifications to the server asynchronously, you need to refresh some parts according to the new state of the server. Which parts do you refresh?

While the answer might seen easy in every single specific case (I’m updating this list/object/grid, so I’ll just refresh it), the general answer is not so simple: your server-side modification might have an impact on other parts of your system.

Consider a typical Facebook-like interface: you have a menu with an inbox, and to the right of that inbox there’s the number of unread messages. On the inbox page, you have a list of messages with a little cross on each message that deletes it through AJAX. The naive thing to do is have that cross update the list of messages, but then deleting an unread message wouldn’t update the menu.

Inevitably, a developer working on an AJAX feature will forget to take into account that some other part of the page that needs to be updated. Or a developer will add some information to every page and forget that some pages need to update that information.

Repeating yourself

Javascript does not benefit from the same clean separation of features into classes, files, packages and namespaces. Also, IDE quality is lacking when compared to other languages. This makes it hard to refactor JS code when duplicate functionality starts to appear.

Let’s consider the asynchronous post situation. In order for that code to work, you need to have fields with identifiers ‘user’ and ‘name’ and some element to initiate the post through an event. This is not encapsulated: if another page needs similar “post user name” functionality, the code will have to be rewritten. In fact, when the code is that small, it’s actually faster to rewrite it than it is to find and call an existing function (not to mention writing that function in the first place).

No refactoring means the code repeats itself. Having two user-name-change pieces of javascript on two different pages means twice as much work to do when you eventually change how that part really works.

Allowing complex behavior to be written in two or three lines is no excuse for letting your code get out of hand: stand firm by the “once, twice, refactor” motto and do not hesitate to turn a three-liner into a ten-line reusable function with appropriate documentation.

PHP Type Checking

Published
by
Victor Nicollet
on August 24, 2009
in Dynamic
. 0 Comments

PHP does not enforce types at compile-time (if anything, because there isn’t a compile time) and runtime checking only happens at the leaves of your source code tree, when you use a PHP function and that function notices one of its arguments is incorrect.

There are of course ways of introducing additional type safety into PHP code, both through development practices and through hints. For instance, you can hard-code checks into function prologues:

function SetUsername($username, $usr_id)
{
  assert (is_string($username));
  assert (is_int($usr_id));
  // ...
}

And, if using class types, you can also use the type hint mechanism in PHP 5 to get automatic warnings:

function FitToWindow(Image $img, Window $window)
{
  // ...
}

There remains the issue of member variables, which are modified and read in many different places. This means a “check the object is in a valid state” function is an useful addition to a class, to be used as a validity check during development to catch any errors as soon as they occur.

I sometimes use the following for my checks:

class Type
{
 public static function Is($value, $type)
 {
   if (func_num_args() > 2) {
     $args = func_get_args();
     array_shift($args);
     return self::Is($value, $args);
   }

   if (is_string($type))
     return self::Is($value, array_filter(explode(' ', $type)));

   if (empty($type))
     return true;

   $first = array_shift($type);

   if ($first == 'null')
     return $value === null || self::Is($value, $type);

   if ($first == 'array') {
     if (!is_array($value))
       return false;
     $next = 0;
     foreach ($value as $key => $val) {
       if ($key != $next++)
         return false;
       if (!self::Is($val, $type))
         return false;
     }
     return true;
   }

   if ($first == 'time')
     return is_int($value) && $value >= 0;

   if ($first == 'hash') {
     if (!is_array($value))
       return false;
     foreach ($value as $val)
       if (!is($val, $type))
         return false;
     return true;
   }

   if (is_callable($first))
     return call_user_func($first, $value) && self::Is($value, $type);

   if (is_callable('is_' . $first))
     return call_user_func('is_' . $first, $value) && self::Is($value, $type);

   if (class_exists($first))
     return($value instanceof $first);

   return false;
 }

 public function checkTypes()
 {
   self::check($this);
 }

 public static function check($obj)
 {
   $class = get_class($obj);
   foreach (get_class_vars($class) as $var => $value)
     if ($var{0} != '_')
       if (!is($obj->$var, $value))
         throw new Exception("Type error: `$class::$var` is not of type `$value`");
  }
}

The typical use is to define a new class, then assign a default value to all type-checked variables: that default value is a type string (or array) that is parsed and verified by the check functions. For instance:

class User
{
  var $id = 'int';
  var $name = 'null string';
  var $media = 'array Media';
  var $friends = 'positive int';
  var $_hash;
}

This would check that the identifier is an integer, that the name is a string or null, that media is an array of instances of the Media class, and that friends is an integer such that is_positive($obj->friends) returns true (assuming you define that function somewhere). The hash variable is unchecked because it starts with an underscore. This has some advantages:

  • Type expressions are shorter than the corresponding assert statements.
  • They go deeper as far as checks go (for instance, arrays also check that all members are of a certain type).
  • They document the code, by explaining in the class definition what the types of the variables are, as opposed to staying in a function.
  • They help with automated testing by allowing the creation of classes with arbitrary values of the chosen type.

This also has disadvantages:

  • This prevents setting an actual default value for the variables.
  • It introduces an artificial naming convention for variables starting with or without underscores.
  • Type-checking arrays or large structures takes time.
  • It’s not detected by documentation generators.
  • Does not play well with private variables.

You’re not a person

Published
by
Victor Nicollet
on August 14, 2009
in Imperative
. 0 Comments

WEEK 1

In this application, every person belongs to exactly one team.

WEEK 4

We need to manage external contractors. We could use the “person” object.

WEEK 5

Hey, we need to assign a team to every person. Let’s create an “external” team.

WEEK 127

Did you see that newspaper article about our company? They say we have an average of 30 people on every team. Do we even have 30-people teams?

Names are short. They can only convey a very limited amount of information. Even worse, that information tends to be different from its meaning in standard English: by declaring in week one that every person belongs to a team, the project designers separated the Application::Person (always in a team) from the English::Person (might be in zero, one or more teams). By week four, this separation vanished from the minds of most of the team. A developer noticed that “English::Contractor is-a English::Person” and mistakenly translated it to “Application::Contractor is-a Application::Person“.

This was the first mistake. Why didn’t he notice?

A positive property is what you can do with a thing.With the Person object, you can store a name, login, password and phone number!This is exactly we you needed! Those positive properties you need that the object doesn’t provide, you can always add them through inheritance or composition, and that’s still less work than implementing everything or having to refactor the code. A negative property is what you cannot do with an object no matter how hard you try. With the Person object, you cannot remain on your own without a team! But our brains are biased to look for positive properties first, and passively ignore negative properties until it’s too late. Positive properties are about the solution solving the problem. Negative properties are about the solution not being applicable.

The second mistake was, by far, the worst. So they finally noticed that negative property that blasted all their model away. And they went on with it, patching the issue by altering the meaning of Application::Team. It originally a project team within the company, it then represented a named group of people that could be a project team or the group of external contractors. This is refactoring: no matter how you look at it, you change the behavior of an object and let it propagate throughout the project, so you better be careful about where it propagates! In this case, they weren’t careful about propagating the change of meaning to the documentation and user interaction part of the project, who mistakenly kept the old meaning of Application::Team. This led to a naive PR team issuing a statement that included the “external” group as if it were a project team.

It’s always helpful to have an anal-retentive person in a group, preferably in a position of authority that lets them veto such changes, and who is vigilant enough to spot that “external” team early on in the design.

The real mistake was allowing a negative property to slip into the design. Negative properties hinder reuse, by definition. Sure, allowing a person to belong to zero-one-many teams is hard on every piece of code that must work on teams, because the writers have to remember to check whether the person has a team in the first place. But it has to be done. Doing it may even bring to light some issues in the original requirements (”So what happens when a person changes teams between the moment team bonuses are computed and the moment they are paid out?”) that would become annoying later on.