HTML Robustness and Query String Parameter Naming

All the way back to 10 years ago, HTML was designed for non-professionals to publish information to the world wide web. The parsing and interpreting rules are loose. Web browsers implemented a lot of heuristics to display as much information as possible.

Now, we have been entered the age of Web 2.0 for a few years. The legacy loosen rules remains. One of the rules I have discovered today is the HTML entity. HTML entity without semicolon will be interpreted as if it has semicolon. For example, &lt will be interpreted as <.

The problem is when writing a web application. You will need to concatenate query string in <a> tag like <a href="list.php?a=x&b=y">. The query string parameters cannot be named as one of those HTML entities. So, <a href="list.php?a=x&lt=y"> does not work. The URL will be interpreted as list.php?a=x<=y. Certainly, no one would use lt as the name of a query string parameter. However, some other common names including euro, copy, pound, cent, uml, not, micro, times and divide will not work.

Developer Friendliness

People usually said that Firefox is more developer friendly than IE or Safari. Currently, there are two embarrassing incidents that flip my perception of Firefox.

1. Text box inside DIV
For some weird reason in Firefox 2, the cursor caret in the text box disappears when the text box is inside a DIV. Web developer has to work with the CSS hack to get the caret back

overflow: hidden; position:fixed;

It is OK if web developer have the total control of the HTML. However, we have web components in .Net and Java Server Face. If such components do not aware of this, user may had a bad experience.

2. Length Required with POST
Content-length is an HTTP header letting the server know how much data it should be expecting in the request. However, Firefox does not send such header to the server with posting with XMLHTTPRequest. Developer need to write it out explicitly

xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded')
xhr.setRequestHeader('Content-length', '0');

XMLHTTPRequest is the key doing AJAX! The HTTP request is usually uncachable when using POST. Why on earth developers need to code it themselves?

I think implementing the standard is important. It relieves the web developers headache. On the other hand, please do not introduce new problem to the developers.

ShineDraw in JavaScript: Kaleidoscope

This is the JavaScript version of Kaleidoscope in ShineDraw. It uses <canvas> and works with Gecko (Firefox, SeaMonkey), Webkit (Google Chrome, Safari) engines.

Performance:

• SeaMonkey - Comsumes a lot of CPU power
• Google Chrome - Renders the Kaleidoscope smoothly.

ShineDraw in JavaScript: Random Expand Rotator

This is the JavaScript version of Random Expand Rotator in ShineDraw.


Implementation Note:
Image scaling and clipping cannot be done at the same time in JavaScript.

ShineDraw in JavaScript: Waving Text

This is the JavaScript version of Waving Text in ShineDraw.

How Bad Can a Bad MPEG Decoder Be?

I have be using VirtualDub for years to encode video files. I found many criticisms VirtualDub is being old. We should use more modern encoder support new file format like MP4. Avidemux seems very popular as it has as GUI. No command line option to be remembered.

I used Avidemux to try to encode a few files and it seems that it could get the job done. Until recently, I found a very unforgivable feature (a.k.a. a big bad bug)! The frame detail got blurred! I thought it was a blurred by the codec at the beginning. However, when I look at the preview screen in Avidemux, the details has been destroyed already comparing with VirtualDub and Media Player Classic (which deinterlaced the frame and still looks good).

We want to minimize the quality loss of the video in many ways like increasing bitrate. Losing the quality while decoding the source file is not acceptable. Please trash it right the way!

Taking Google Maps API Offline

Sometimes, we want to utilize the some API for other work. Google Maps API is a very good example. You may want to use it as a photo viewer. Like allowing user to zoom-in and zoom-out, tagging, etc. However, since the API needs to validate the key online, you cannot take it as a standalone application where people can use offline.

Now, Google Maps API is a JavaScript API which we can download the script from the web. It just needs a bit hacking to get the API offline:
1. Highly obfuscated code
2. Dynamic module loading
3. Key verification

The obfuscated code can be ignored since you do not need to understand everything. All you need is finding the key statements that involve loading resources online (like module loading, image loading, etc). You can use a formatter to beautify it a little bit and decode it mentally.

The key verification is the most easiest part. Like all hacking, find the verification function and return a true will be good.

The process can be done within an hour for an experienced programmer.

With a good packager, we can have a really nice image mosaic viewer offline.

Microsoft Servers eBooks

There has been some times that we want to play with some server but have no idea how to start with. Here are some of the Microsoft Servers eBooks for installation.

• SQL Server 2005 Books Online (September 2007)
• BizTalk Server 2006 R2 Installation and Upgrade Guides
• Installation Guide for Microsoft Office SharePoint Server 2007
• Deploying a Standard Exchange Server 2007 Organization
• Microsoft Dynamics CRM 4.0 Implementation Guide
• Deployment Guide for PerformancePoint Server 2007

Receiving AS2 MDN in BizTalk Orchestration

Having an AS2 port setup in an orchestration is easy to configure a one-way send port. However, configure a request-response send-port is a bit tricky. You probably know the message schema of the send port but you have no idea of the schema of an MDN.

You may attempt to use System.Object, System.String, or even create an MIME message type with two parts. However, none of these options work. Going through the schemas under the BizTalk EDI Application in Administration Console, you only see the property schema of EdiIntAs.EdiIntProperties and no sign for other schema for MDN.

At this moment, you will want to have a type that receiving any message. Microsoft.XLANGs.BaseTypes.Any is the savior. You can choose it from Schema > Select from referenced assembly...

Moving to .Net 3.0/3.5

WF, WCF, WPF, and LINQ are going to be pushed by Microsoft. To be honest, the framework is just unbeatable by other current programming frameworks. Adobe has the equivalent technologies but they are more on documents only and have not fully integrated together yet. Open source camp is too scattered. It is hard to imagine it can response quickly to the .Net 3.5 wave.

A good artical about C# and VB.Net

Despite that the article has been written for a long time, it is still a good article to read.
http://www.codeproject.com/dotnet/CSharpVersusVB.asp

Thoughts on Silverlight

Microsoft is pushing Silverlight so hard these days. As the official release is approaching, the adoption rate will be low if it use the same installation method of beta. A better off will be Flash like installation. Everything can be installed automatically and no restart of browser. The real frictionless environment.

I could provision it could be a big success when Microsoft solves this particular problem.

Software Design Corrosion

No matter how good is the design of a piece of software, the design corrodes proportional to the the time gathered the user requirements. As time goes by, the user requirements change. The design of the software become less oriented to the new user requirements. The programmers may feel the limitation of the original design. However, they have the stick with the design and trying to change the fewest thing to accomplish the goal - the user requirement.

Hence, cheap hacks and workarounds are introduced. These hacks and workarounds may not hurt the maintainability in short term. However, since the design may not consider the effect of these hacks and workarounds, some other factor may get affected. For example, security may get compromised.

Take XmlHttpRequest as and example, it is designed for retrieving XML or simple text data. However, people use it to retrieve JSON that allows executable JavaScript.

Even large enterprise applications are vulnerable to the corrosion. The architecture design of enterprise application is usually coarse grained due to the time and resource limitation in the business world. Hence, hacks and workarounds are very easy to introduced among a team of programmers. Since enterprise applications are large, it is not quite possible for the architects or the analysts to review the code. Thus, the design corrosions are easily occurred.

There is no easy solution to the phenomena. Adding extra resources may slow down the effect. However, it cannot be completely removed.

Some Thoughts on Ruby

Ruby on Rails (RoR) seems very promising. However, not every web applications are CRUD applications. It is very easy to have the similar frameworks to implement it in Java or PHP. Even Ruby got the attention now, it will like all buzz diminishing at the end.

Let's see how I reach the conclusion from the successful platform.

Java

1. Cross-platform
2. Language of C family
3. Huge number of libraries
4. Backed by big corp
5. Excellent Javadocs API
   
6. Nice IDEs
   

.Net

1. Support multiple languages
2. Medium size .Net libraries but able to talk with legacy DLLs
   
3. Backed by big corp
4. Very nice MSDN
   
5. Very nice IDE

PHP

1. Cross-platform
2. Morph from C family
   
3. Large size of libraries
4. Interactive PHP Manual (i.e. users can add the comments providing examples, tricks, etc)
   

To make a language/platform success, the language family, the size of libraries, and the documentation are very critical. Perl, for example, does have the whole CPAN libraries and quite well documented. However, the cryptic syntax scares off so many people. That's how PHP became more popular than Perl.

Java and .Net have the advantages because they are backed by big corporations and they have the resource to get the documentation written nicely. PHP does not have the financial advantage but it starts off very simple at the beginning. Documentation can be written and updated bit by bit.

Now, Ruby is not a language belongs to C family. I believe that the size of the libraries cannot be larger than CPAN. The library should be well written. However, it does not have a nice IDE. It does not have big corporation support. It only has a framework letting programmer to get the easy job done. As I mentioned in the first paragraph, the framework can be written in Java/PHP easily. It is not a good advantage. Is it going to success?

Alternatively, I think Java and server-side JavaScript (Rhino engine) can have the synergy. Simply speaking, RoR reads database and generates the data model, the controllers, and the views. Java can read the database (from MySQL to Oracle) to generate the data model and the controllers in JavaScript (for the sake of flexibility) and use a template engine (JSP, Velocity, Freemarker) to generate the view. Since ALL web programmers know JavaScript already, they can REUSE what they have already known. With some simple documentation to let the programmers know how to get the job done, it is very easy to get adopted.

P.S. JavaScript success factors:

1. Cross-platform
2. Language of C family
   
3. Large size of libraries (esp. after the AJAX boom)
   
4. No official documentation but references are just a click away from the search engine
   

Screwing up HTTP Session for Servlet/JSP

Prerequisites:
1. Browsers under Mozilla project (e.g. Firefox, SeaMonkey)
2. Enable cookies
3. Tomcat (probably all Java web containers)

Procedure:
1. Write a page that use back-slashes as the directory seperator (e.g. http://www.abc.com\app\page.jsp)
2. Include this page with absolute path in all pages

Why:
1. The browser encode back-slashes as %5C. So, http://www.abc.com\app\page.jsp will be encoded as http://www.abc.com%5Capp%5Cpage.jsp
2. The browser tries to look up the cookies for the domain. However, the browser think that the domain name is www.abc.com%5Capp%5Cpage.jsp instead of www.abc.com
3. The browser cannot find the cookies for the domain. So, it does not send the cookies to the browser.
4. When cookies is enabled, Java Servlet and JSP uses it to store the session ID
5. Since the cookies was not found in the request header, the web container creates a new session and send to session ID with Set-Cookie header.
6. The browser uses the new session ID for the next request.
7. As a result, the web container creates new session for each browser request and the sessions are screwed up.

What about other browsers? I think IE is smart enough to replace all back-slashes to slashes before it does further processing. I don't know about others though...

You may want to ask why you would use back-slash. My answer is "No, you may not but your staff or your outsource partner may." In a workflow engine we are studying, we found the following JavaScript:
document.write("<iframe src='<%=request.getContextPath()%>\\app\\page.jsp'>")

My colleague asked for help about why the session does not work in Firefox. Since the code was written by different parties, we know nothing about the workflow engine. I alone used 6 man-hours to fix the BUG. I think I know fairly well on HTTP and Java Servlet. I tried many possible solutions like finding the HttpSession.invalidate(), session.removeAttribute(), appending the jsessionid, etc. Finally, I used a packet sniffer to dig out the pattern that changes the jsessionid cookie and recognize the weired %5C URL in a GET command.

Adobe Flex Push

There are ready to go assemblers in FDS samples application provided for you to develop your own Flex application with FDS. It is quite impressive that when a client update a piece of data, the FDS can synchronize the data to other Flex client without extra coding. However, what if you only want to push some changes to the client?

Calling update in assembler may be a good option. Using the DataServiceTransaction could be better:

DataServiceTransaction tx = DataServiceTransaction.begin(false);
Rate rate = ... // get the rate from some streaming data source

// do any necessary calculation
tx.updateItem("fds.rate", rate, null, null);
tx.commit();

This snippet should get the job done on the server side. On the client side, you need to handle the FDS message:

 <mx:script>
public function handleMessage(event):void {
  if (rate.id == event.message.body[2].id) {
    rate = event.message.body[2];
  }
}
</mx:script>
<mx:label text="{rate.value}"/>
<mx:dataservice id="dsRate" destination="fds.rate" message="handleMessage(event)" />

Base64, MHTML, and IE

In HTML 4.0, binary objects (e.g. image) can be inlined with Base64 encoding. However, Internet Explorer totally ignores the standard. Even in IE7, base64 image is still not supported.

Two solutions proposed by Dean Edwards and Luiz Angelo De Luca are trying to solve the problem. Both of them are implemented in PHP. Base64Html is a Java implementation of Luiz Angelo De Luca's solution: convert the HTML into MHTML (HTML Mail format).

Luiz Angelo De Luca's solution can be genernalize to the following steps:

1. Buffer the output and add a function callback
2. If the browser is IE, set the HTTP header to MHTML
3. Convert the HTML to MHTML
   

Luiz Angelo De Luca has implemented it in a single PHP. In Java, we make use of HttpServletResponseWrapper (for #1), Serlvet Filter (for #2), and Apache Jakarta Commons Email API (for MHTML construction). Source code and example web app are available at: http://www.shaneng.net/Main/Base64Html

3G is Destinated to Failure

When 3G providers are launching the 3G services, I think that it will not success. The currently marketed feature of 3G is the video call. However, I would not use a 3G phone because of the video. First, the video resolution sucks. Second, I don't want the person (esp. girl) I am talking to know where I am.

Now the market shift the focus from the video call to video content. It won't work for me neither. First, why on earth I want to watch video on the street? Listening music or radio is OK but watching video? Nah...

Secondly, it is way too expensive. Well, bandwidth and information have their price but not for video entertainment. Moreover, as digital TV become more popular, there are a lot of free entertainment signals for the public already. If we could decode the signal with a TV box, why can't we decode it with a mobile phone?

Conclusion: 3G is destinated to failure

Dissection of Technologies in JavaScript RIA

In the last entry, I have summarized choices we have while writing a JavaScript RIA (or AJAX application). They are under these seven categories:

1. Communication Model
2. Communication Style
3. Push Communication Style
4. Transportation Protocols
5. Data Format
6. UI Rendering
7. UI Update Style

These categories are actually order in a particular user request:

1. Is it an asychronous or a synchronous?
2. Use iFrame or XHR?
3. Use iFrame or HTTP 1.1 chunked for push data?
4. Data is transmitted with HTTP, XML-RPC, or SOAP?
5. Data is returned in what format (plain text, HTML, JSON, Base64, XML)?
6. The data is the rendered UI or the data values for the client to render the UI?
   
7. Use innerHTML or use DOM to maniuplate the UI?

The combination of the technologies can be summarized in the following graph:

To make any design decision, we may highlight the nodes and paths in the graph in top-down manner. For example, in the case of original AJAX defintion, we may hightlight Asychronous, XHR, HTTP/REST, XML, and Client Side Rendering:

The UI update style is undetermined up to this moment. We can make a complete decision before sending the design to programmers. Designers and programmers may have different preference of AJAX approach. With this simple graph, designers can communicate and synchronize with programmers more effectively. The architect can also use the graph to generate the implementation blueprints for programmers to follow so that more maintainable code can be expected.

The way to JavaScript RIA is paved with good technologies

As we are working with the Shanghai team, we found that they have different concepts about JavaScript RIA (also known as AJAX). Hong Kong team tends to have a better structure and more graceful way to partition the layers. On the other hand, Shanghai team tends to have less structure but faster development speed.

This leads to the classical design question about the tradeoff between development time and maintenance time.

For this reason, I listed out all possible combinations of different technologies and styles so that we can make the design decision from the beginning of the design stage in the future. The explanations will be added soon. To summarized, they are:

1. Communication Model
• Synchronous
• Asynchronous (AJAX)
2. Communication Style
• iFrame
• XMLHTTPRequest (XHR)
3. Push Communication Style
• iFrame
• HTTP 1.1 chunked
4. Transportation Protocols
• HTTP / REST
• XML-RPC
• SOAP
5. Data Format
• Plain Text
• HTML
• XML
• JSON
• Base64
6. UI Rendering
• Server Side Rendering
• Client Side Rendering
7. UI Update Style
• innerHTML Substitution
• DOM Manipulation