Software Design Corrosion

No matter how good is the design of a piece of software, the design corrodes proportional to the the time gathered the user requirements. As time goes by, the user requirements change. The design of the software become less oriented to the new user requirements. The programmers may feel the limitation of the original design. However, they have the stick with the design and trying to change the fewest thing to accomplish the goal - the user requirement.

Hence, cheap hacks and workarounds are introduced. These hacks and workarounds may not hurt the maintainability in short term. However, since the design may not consider the effect of these hacks and workarounds, some other factor may get affected. For example, security may get compromised.

Take XmlHttpRequest as and example, it is designed for retrieving XML or simple text data. However, people use it to retrieve JSON that allows executable JavaScript.

Even large enterprise applications are vulnerable to the corrosion. The architecture design of enterprise application is usually coarse grained due to the time and resource limitation in the business world. Hence, hacks and workarounds are very easy to introduced among a team of programmers. Since enterprise applications are large, it is not quite possible for the architects or the analysts to review the code. Thus, the design corrosions are easily occurred.

There is no easy solution to the phenomena. Adding extra resources may slow down the effect. However, it cannot be completely removed.

Some Thoughts on Ruby

Ruby on Rails (RoR) seems very promising. However, not every web applications are CRUD applications. It is very easy to have the similar frameworks to implement it in Java or PHP. Even Ruby got the attention now, it will like all buzz diminishing at the end.

Let's see how I reach the conclusion from the successful platform.

Java

1. Cross-platform
2. Language of C family
3. Huge number of libraries
4. Backed by big corp
5. Excellent Javadocs API
   
6. Nice IDEs
   

.Net

1. Support multiple languages
2. Medium size .Net libraries but able to talk with legacy DLLs
   
3. Backed by big corp
4. Very nice MSDN
   
5. Very nice IDE

PHP

1. Cross-platform
2. Morph from C family
   
3. Large size of libraries
4. Interactive PHP Manual (i.e. users can add the comments providing examples, tricks, etc)
   

To make a language/platform success, the language family, the size of libraries, and the documentation are very critical. Perl, for example, does have the whole CPAN libraries and quite well documented. However, the cryptic syntax scares off so many people. That's how PHP became more popular than Perl.

Java and .Net have the advantages because they are backed by big corporations and they have the resource to get the documentation written nicely. PHP does not have the financial advantage but it starts off very simple at the beginning. Documentation can be written and updated bit by bit.

Now, Ruby is not a language belongs to C family. I believe that the size of the libraries cannot be larger than CPAN. The library should be well written. However, it does not have a nice IDE. It does not have big corporation support. It only has a framework letting programmer to get the easy job done. As I mentioned in the first paragraph, the framework can be written in Java/PHP easily. It is not a good advantage. Is it going to success?

Alternatively, I think Java and server-side JavaScript (Rhino engine) can have the synergy. Simply speaking, RoR reads database and generates the data model, the controllers, and the views. Java can read the database (from MySQL to Oracle) to generate the data model and the controllers in JavaScript (for the sake of flexibility) and use a template engine (JSP, Velocity, Freemarker) to generate the view. Since ALL web programmers know JavaScript already, they can REUSE what they have already known. With some simple documentation to let the programmers know how to get the job done, it is very easy to get adopted.

P.S. JavaScript success factors:

1. Cross-platform
2. Language of C family
   
3. Large size of libraries (esp. after the AJAX boom)
   
4. No official documentation but references are just a click away from the search engine
   

Screwing up HTTP Session for Servlet/JSP

Prerequisites:
1. Browsers under Mozilla project (e.g. Firefox, SeaMonkey)
2. Enable cookies
3. Tomcat (probably all Java web containers)

Procedure:
1. Write a page that use back-slashes as the directory seperator (e.g. http://www.abc.com\app\page.jsp)
2. Include this page with absolute path in all pages

Why:
1. The browser encode back-slashes as %5C. So, http://www.abc.com\app\page.jsp will be encoded as http://www.abc.com%5Capp%5Cpage.jsp
2. The browser tries to look up the cookies for the domain. However, the browser think that the domain name is www.abc.com%5Capp%5Cpage.jsp instead of www.abc.com
3. The browser cannot find the cookies for the domain. So, it does not send the cookies to the browser.
4. When cookies is enabled, Java Servlet and JSP uses it to store the session ID
5. Since the cookies was not found in the request header, the web container creates a new session and send to session ID with Set-Cookie header.
6. The browser uses the new session ID for the next request.
7. As a result, the web container creates new session for each browser request and the sessions are screwed up.

What about other browsers? I think IE is smart enough to replace all back-slashes to slashes before it does further processing. I don't know about others though...

You may want to ask why you would use back-slash. My answer is "No, you may not but your staff or your outsource partner may." In a workflow engine we are studying, we found the following JavaScript:
document.write("<iframe src='<%=request.getContextPath()%>\\app\\page.jsp'>")

My colleague asked for help about why the session does not work in Firefox. Since the code was written by different parties, we know nothing about the workflow engine. I alone used 6 man-hours to fix the BUG. I think I know fairly well on HTTP and Java Servlet. I tried many possible solutions like finding the HttpSession.invalidate(), session.removeAttribute(), appending the jsessionid, etc. Finally, I used a packet sniffer to dig out the pattern that changes the jsessionid cookie and recognize the weired %5C URL in a GET command.

Adobe Flex Push

There are ready to go assemblers in FDS samples application provided for you to develop your own Flex application with FDS. It is quite impressive that when a client update a piece of data, the FDS can synchronize the data to other Flex client without extra coding. However, what if you only want to push some changes to the client?

Calling update in assembler may be a good option. Using the DataServiceTransaction could be better:

DataServiceTransaction tx = DataServiceTransaction.begin(false);
Rate rate = ... // get the rate from some streaming data source

// do any necessary calculation
tx.updateItem("fds.rate", rate, null, null);
tx.commit();

This snippet should get the job done on the server side. On the client side, you need to handle the FDS message:

 <mx:script>
public function handleMessage(event):void {
  if (rate.id == event.message.body[2].id) {
    rate = event.message.body[2];
  }
}
</mx:script>
<mx:label text="{rate.value}"/>
<mx:dataservice id="dsRate" destination="fds.rate" message="handleMessage(event)" />

Base64, MHTML, and IE

In HTML 4.0, binary objects (e.g. image) can be inlined with Base64 encoding. However, Internet Explorer totally ignores the standard. Even in IE7, base64 image is still not supported.

Two solutions proposed by Dean Edwards and Luiz Angelo De Luca are trying to solve the problem. Both of them are implemented in PHP. Base64Html is a Java implementation of Luiz Angelo De Luca's solution: convert the HTML into MHTML (HTML Mail format).

Luiz Angelo De Luca's solution can be genernalize to the following steps:

1. Buffer the output and add a function callback
2. If the browser is IE, set the HTTP header to MHTML
3. Convert the HTML to MHTML
   

Luiz Angelo De Luca has implemented it in a single PHP. In Java, we make use of HttpServletResponseWrapper (for #1), Serlvet Filter (for #2), and Apache Jakarta Commons Email API (for MHTML construction). Source code and example web app are available at: http://www.shaneng.net/Main/Base64Html

3G is Destinated to Failure

When 3G providers are launching the 3G services, I think that it will not success. The currently marketed feature of 3G is the video call. However, I would not use a 3G phone because of the video. First, the video resolution sucks. Second, I don't want the person (esp. girl) I am talking to know where I am.

Now the market shift the focus from the video call to video content. It won't work for me neither. First, why on earth I want to watch video on the street? Listening music or radio is OK but watching video? Nah...

Secondly, it is way too expensive. Well, bandwidth and information have their price but not for video entertainment. Moreover, as digital TV become more popular, there are a lot of free entertainment signals for the public already. If we could decode the signal with a TV box, why can't we decode it with a mobile phone?

Conclusion: 3G is destinated to failure

Dissection of Technologies in JavaScript RIA

In the last entry, I have summarized choices we have while writing a JavaScript RIA (or AJAX application). They are under these seven categories:

1. Communication Model
2. Communication Style
3. Push Communication Style
4. Transportation Protocols
5. Data Format
6. UI Rendering
7. UI Update Style

These categories are actually order in a particular user request:

1. Is it an asychronous or a synchronous?
2. Use iFrame or XHR?
3. Use iFrame or HTTP 1.1 chunked for push data?
4. Data is transmitted with HTTP, XML-RPC, or SOAP?
5. Data is returned in what format (plain text, HTML, JSON, Base64, XML)?
6. The data is the rendered UI or the data values for the client to render the UI?
   
7. Use innerHTML or use DOM to maniuplate the UI?

The combination of the technologies can be summarized in the following graph:

To make any design decision, we may highlight the nodes and paths in the graph in top-down manner. For example, in the case of original AJAX defintion, we may hightlight Asychronous, XHR, HTTP/REST, XML, and Client Side Rendering:

The UI update style is undetermined up to this moment. We can make a complete decision before sending the design to programmers. Designers and programmers may have different preference of AJAX approach. With this simple graph, designers can communicate and synchronize with programmers more effectively. The architect can also use the graph to generate the implementation blueprints for programmers to follow so that more maintainable code can be expected.

The way to JavaScript RIA is paved with good technologies

As we are working with the Shanghai team, we found that they have different concepts about JavaScript RIA (also known as AJAX). Hong Kong team tends to have a better structure and more graceful way to partition the layers. On the other hand, Shanghai team tends to have less structure but faster development speed.

This leads to the classical design question about the tradeoff between development time and maintenance time.

For this reason, I listed out all possible combinations of different technologies and styles so that we can make the design decision from the beginning of the design stage in the future. The explanations will be added soon. To summarized, they are:

1. Communication Model
• Synchronous
• Asynchronous (AJAX)
2. Communication Style
• iFrame
• XMLHTTPRequest (XHR)
3. Push Communication Style
• iFrame
• HTTP 1.1 chunked
4. Transportation Protocols
• HTTP / REST
• XML-RPC
• SOAP
5. Data Format
• Plain Text
• HTML
• XML
• JSON
• Base64
6. UI Rendering
• Server Side Rendering
• Client Side Rendering
7. UI Update Style
• innerHTML Substitution
• DOM Manipulation

Building a Source Code Generator II

Last article described the basic infrastructure of writing a source code generator. The design is now implemented in Codejen Framework and available in SourceForge.net.

Building a Source Code Generator I

Recall the following components to build a source code generator:

1. A model that describe your application - you don't need to define it yourself, you can use XMI or XML schema as the reference model. The only problem is how you parse them to the object model.
   
2. A template engine - Velocity or Freemaker are good template engines in Java.
   
3. Templates that corresponding to the model - Generate code Java, PHP, C#, etc.
   
4. A configuration of templates - There would be configurations for desktop app, web app, RIA, or web services, etc.
   
5. A parser that parse the configuration - If the configuration is an XML file, Apache's commons-digester library can help.
6. A driver to glue up everthing.
7. Optionally, a validator for the template is good - JavaCC or ANTRL for syntax validation. For the extreme, use a compiler to verify the correctness of the generated code.
   

The diagram on the left included the classes designed for #4 to #7.

• CodeGenerator - the parser of the configuration file and the driver gluing up everything.
  
• TemplateConfiguration - an list of template that can stores the properties specified in the XML file.
• Template - a class representing the state of a template.
• TemplateProcessor - processor that may perform validation or beautification of the generated code.

With this general source code generator, it can be adapted to use with other CASE tools (e.g. Visual-Paradigm).

Example configuration XML:

<profile>
<property key="asdf" value="1" />
<property key="templateDir" value="x ${asdf}" />
<template class="org.sf.codejen.test.TestTemplate" a="${templateDir}">
 <postProcessorFactory class="org.sf.codejen.test.TestProcessorFactory" b="jkl;" />
</template>
</profile>

The required attributes are

• profile/property - key and value
• profile/template - class
• profile/template/postProcessorFactory - class

All other attributes are optional depends on the class design.
Also, if you are familiar with ANT, you can use ${propKey} style to access the property value.

How to build a source code generator

1. A model that describe your application - you don't need to define it yourself, you can use XMI or XML schema as the reference model. The only problem is how you parse them to the object model.
   
2. A template engine - Velocity or Freemaker are good template engines in Java.
   
3. Templates that corresponding to the model - Generate code Java, PHP, C#, etc.
   
4. A configuration of templates - There would be configurations for desktop app, web app, RIA, or web services, etc.
   
5. A parser that parse the configuration - If the configuration is an XML file, Apache's commons-digester library can help.
   
6. A driver to glue up everthing.
7. Optionally, a validator for the template is good - JavaCC or ANTRL for syntax validation. For the extreme, use a compiler to verify the correctness of the generated code.
   

Web 2.0 is Just a Milestone

After reading some articles and blogs, I found that Web 2.0 is not really new thing. It's just a milestone that saying the internet has evolved into new phases. People are not working alone but collaborated. XML and HTTP make different platforms more integrated comparing to the DCOM and COBRA. Thanks to AJAX, web applications are almost as dynamic as desktop applications. Whenever you go, whereever your are. You can access the application with your electronic devices (mobile phone, PC, PDA, etc). No more deployment and update patches is need.

If Web 2.0 is a milestone, what else we can do with it? Just celerbrate and get back to work! Have we rearched the milestone yet? Not really! Let's work harder for it.

The Future of Programming

I share the same vision about programming with Microsoft. DSM will rule and MDA cannot dominate the market. So, does it imply programming will no longer exist if we define enough DSL? The answer is no.

There aren't "enough" DSL in the future. At least, there will be two camps of DSM tools, Microsoft (Software Factories) and Eclipse (EMF+VEF). For MS, sure it will be only .NET platform. For Eclipse, however, it will be for the whole all platforms. For different languages, UI, and DBMS. There are infinite combinations for us to define.

The future of programming is programming the DSL for user to define it's own application on the system specifications.

JavaScript Obfuscator

I have recently written an obfuscator for JavaScript in Java. It is available at:
http://www.shaneng.net/

It exploit some feature of JavaScript so that I don't need to understand the content of JavaScript. To start, the obfuscator just "blindly" replace the variables without leading dot (e.g. window.parent has the dot). Then, I managed to remove the comments. Later, it also encode character in the string literals into \uXXXX or \XXX format.

In the current version, I refactor it so that it is more managable. In the future, I would like to handle the source code with E4X syntax.

JavaScript: setInterval() problem

Have you been trying to write some class like this:

function y() {
  this.x = 0;
  this.start = function () {
      this.interval = setInterval(this.run, 1000);
  };

  this.run = function () {
      this.x++;
      debug(this.x); // or some other things you want to do
  };

  this.stop = function () {
      clearInterval(this.interval);
  };
}

You may attempt to refer the x in the run() function. However, in JavaScript, the scope of the function becomes the window after running setInterval() (or setTimeout()).

To solve the problem, the code be written like this:

function y() {
   this.x = 0;
   var self;

   this.start = function () {
       self = this;
       this.interval = setInterval(run, 1000);
   };

   function run() {
       self.x++;
       debug(self.x);
   }

   this.stop = function () {
       clearInterval(this.interval);
       self = undefined;
   };
}

Since using the self variable will create a cyclic reference, it should be cleaned up properly. Otherwise, the garbage collector of the script interpreter(s) which use(s) reference count will not be able to collect the instance.

Update (2009-03-29)
Thanks to the comment by dedogs, we could actually write the code like this without the cyclic reference:

function y() {
  this.x = 0;
  this.start = function () {
      var self = this;
      this.interval = setInterval(function(){ self.run(); }, 1000);
  };

  this.run = function () {
      this.x++;
      debug(this.x); // or some other things you want to do
  };

  this.stop = function () {
      clearInterval(this.interval);
  };
}

AJAX: XMLHTTPRequest Event Handling

Most of the XMLHTTPRequest uses the following implementation:

function run1() {
  xmlhttp = new XMLHTTPRequest();
  xmlhttp.onreadystatechange = processStateChange;
  ...
}

function processStateChange() {
  if (xmlhttp.readyState == 4) ...
}

The problem is, when you have many events to handle, the variable xmlhttp in run1 may have assigned to different instances by different functions (e.g. run2, run3, etc). If you rely on this code, no one can gaurentee that your code will run correctly.

The resolve this problem you can try to write the code in this way:

function run1() {
  var xmlhttp = new XMLHTTPRequest();
  xmlhttp.onreadystatechange = function() { ... }
}

If you insist to write the handle in a function, here is another style of writing it:

function run1() {
  var xmlhttp = new XMLHTTPRequest();
  xmlhttp.onreadystatechange = function() {
    processStateChange(xmlhttp);
  }
  ...
}

function processStateChange(xmlhttp) { ... }