MXHR

Digg is creating a multipart capable XHR library. Good try but not very useful. Here is why:

1. MXHR response is not compressed by default.
   This is illustrated very well in their text only demo. MXHR stream is always slower than normal mode. I opened up the packet sniffer. The normal stream is compressed while the MXHR is not.
   
2. Data URI scheme is broken.
   The web is broken by IE for a long time. IE6 and IE7 will be still around for a few years. Image data cannot be served in cross-browser manner. The same performance can be achieved by spriting: putting the tiny-images into one image (overhead is small if the image is large enough), preload the combined imaged, and use background-position to get the coordination.
   
3. Multipart is not streaming
   Well, it is under DUI.Stream namespace. However, it is not streaming. The server is packing up the data into one response. The library is processing the data on-the-fly but the received data is accumulating.
   1. If the size of the response is not large (i.e. can be stream in a couple seconds), enveloping the message in JSON or XML could be easier.
   2. If the size of the response is large (i.e. comet style), the responseText is going to become very large. We call it memory leak.
   

HTTP is broken because it is not intended to serve for Web 2.0. That's why there are so many facilities in CSS and JavaScript to patch the protocol. MXHR is definitely not the answer.

E4X Alternative - JSOM

I have been expecting the popularity of E4X for a long time. However, as long as Internet Explorer is the most popular browser, "new" standards will never get popular (Microsoft is suffered from Windows XP legacy, too). Instead of waiting for E4X, some developers use XPath to query the DOM nodes while the other traverse the DOM node by node. However, it would be good if we could access XML content in JavaScript Object Model (JSOM):

xml='<invoice date="01-20-2000" number="123">' +
    '  <address country="US">' +
    '    <name>John Smith</name>' +
    '    <street>123 George St.</street>' +
    '    <city>Mountain View</city>' +
    '    <state>CA</state>' +
    '    <zip>94041</zip>' +
    '  </address>' +
    '</invoice>';
invoice = JSOM(xml);
alert(invoice.$number);
alert(invoice.address.name);

There is a project, IEE4X, in SourceForge for the same purpose but it does not work quite well for a few cases. So, I grabbed the idea and implement the JSOM library myself. The code can be download at Shane's Shelf.

XML vs. JSON - Size Comparison

I was wondering how compact is JSON comparing to XML. I grabbed an arbitrary XML from the web and convert it to an equivalent JSON format. It turns out that JSON is around two-third of the XML size.

XML: 596 bytes

JSON: 402 bytes

<?xml version="1.0"?> <message> <header> <to>companyReceiver</to> <from>companySender</from> <type>saveInvoice</type> </header> <info> <saveInvoice> <invoice date="01-20-2000" number="123"> <address country="US"> <name>John Smith</name> <street>123 George St.</street> <city>Mountain View</city> <state>CA</state> <zip>94041</zip> </address> <billTo country="US"> <name>Company A</name> <street>100 Main St.</street> <city>Washington</city> <state>DC</state> <zip>20015</zip> </billTo> <items> <item number="1"> <name>IBM A20 Laptop</name> <quantity>1</quantity> <USPrice>2000.00</USPrice> </item> </items> </invoice> </saveInvoice> </info> </message>

{ header:{ to:"companyReceiver", from:"companySender", type:"saveInvoice” }, info:{ saveInvoice:{ invoice:{date:"01-20-2000,number:"123", address:{country:"US", name:"John Smith", street:"123 George St.", city:"Mountain View", state:"CA", zip:"94041" }, billTo:{country="US", name:"Company A", street:"100 Main St.", city:"Washington", state:"DC", zip:"20015" }, items:[ { name:"IBM A20 Laptop", quantity:1, USPrice:2000.00 } ] } } } }

Secure WCF Services with Authentication Service

We can use WCF Authentication Service to authenticate users with ASP.NET membership provider. However, other WCF services are not protected by the authentication service out-of-the-box. That is, WCF services is not using ASP.NET forms authentication.

Fortunately, it is not hard to enable it. The magic point is set the HttpContext.Current.User in Global.asax

public class Global : System.Web.HttpApplication
  {
      // other methods snipped...
      protected void Application_AuthenticateRequest(object sender, EventArgs e)
      {
          HttpCookie ticketCookie = Context.Request.Cookies[FormsAuthentication.FormsCookieName];
          if (null == ticketCookie)
          {
              return;
          }

          FormsAuthenticationTicket ticket = FormsAuthentication.Decrypt(ticketCookie.Value);
          if (null != ticket)
          {
              HttpContext.Current.User = new GenericPrincipal(new FormsIdentity(ticket), null);
          }
      }
  }

In the service you want to protect, set the requirement mode to allowed or required.

    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
   public class PrimeService : IPrimeService

Then, throw in the following checking at the beginning of the method.

            if (!HttpContext.Current.User.Identity.IsAuthenticated)
          {
              throw new FaultException<SecurityAccessDeniedException>(new SecurityAccessDeniedException());
          }

That would be good enough for Silverlight client. For .Net WCF client, you need to handle the HTTP cookies by yourself (Authentication Service is using the authentication ticket in cookies). Detail discussion can be found in the article in Shane's Shelf.

Silverlight Multi-part File Upload Form Post

Silverlight not actually very web friendly. It lacks of built-in support for the current HTML form post protocol. So, we have to write it our own. I have written an extension class and two serializer classes, one for normal form post (DataContractQueryStringSerializer) while the other for multipart upload form post (DataContractMultiPartSerializer).

If you only want to have the working code, just copy the code below. Detail explanation is available at Multi-Part Form Post in Shane's Shelf

  public static class Extensions
  {
      public static void PostFormAsync(this HttpWebRequest request, object parameters, AsyncCallback callback)
      {
          request.Method = "POST";
          request.ContentType = "application/x-www-form-urlencoded";
          request.BeginGetRequestStream(new AsyncCallback(asyncResult =>
          {
              Stream stream = request.EndGetRequestStream(asyncResult);
              DataContractQueryStringSerializer ser = new DataContractQueryStringSerializer();
              ser.WriteObject(stream, parameters);
              stream.Close();
              request.BeginGetResponse(callback, request);
          }), request);
      }

      public static void PostMultiPartAsync(this HttpWebRequest request, object parameters, AsyncCallback callback)
      {
          request.Method = "POST";
          string boundary = "---------------" + DateTime.Now.Ticks.ToString();
          request.ContentType = "multipart/form-data; boundary=" + boundary;
          request.BeginGetRequestStream(new AsyncCallback(asyncResult =>
          {
              Stream stream = request.EndGetRequestStream(asyncResult);

              DataContractMultiPartSerializer ser = new DataContractMultiPartSerializer(boundary);
              ser.WriteObject(stream, parameters);
              stream.Close();
              request.BeginGetResponse(callback, request);
          }), request);
      }
  }

  public class DataContractQueryStringSerializer
  {
      public void WriteObject(Stream stream, object data)
      {
          StreamWriter writer = new StreamWriter(stream);
          if (data != null)
          {
              if (data is Dictionary<string, string>)
              {
                  foreach (var entry in data as Dictionary<string, string>)
                  {
                      writer.Write("{0}={1}&", entry.Key, entry.Value);
                  }
              }
              else
              {
                  foreach (var prop in data.GetType().GetFields())
                  {
                      foreach (var attribute in prop.GetCustomAttributes(true))
                      {
                          if (attribute is DataMemberAttribute)
                          {
                              DataMemberAttribute member = attribute as DataMemberAttribute;
                              writer.Write("{0}={1}&", member.Name ?? prop.Name, prop.GetValue(data));
                          }
                      }
                  }
                  foreach (var prop in data.GetType().GetProperties())
                  {
                      if (prop.CanRead)
                      {
                          foreach (var attribute in prop.GetCustomAttributes(true))
                          {
                              if (attribute is DataMemberAttribute)
                              {
                                  DataMemberAttribute member = attribute as DataMemberAttribute;
                                  writer.Write("{0}={1}&", member.Name ?? prop.Name, prop.GetValue(data, null));
                              }
                          }
                      }
                  }
              }
              writer.Flush();
          }
      }
  }

  public class DataContractMultiPartSerializer
  {
      private string boundary;
      public DataContractMultiPartSerializer(string boundary)
      {
          this.boundary = boundary;
      }

      private void WriteEntry(StreamWriter writer, string key, object value)
      {
          if (value != null)
          {
              writer.Write("--");
              writer.WriteLine(boundary);
              if (value is FileInfo)
              {
                
                  FileInfo f = value as FileInfo;
                  writer.WriteLine(@"Content-Disposition: form-data; name=""{0}""; filename=""{1}""", key, f.Name);
                  writer.WriteLine("Content-Type: application/octet-stream");
                  writer.WriteLine("Content-Length: " + f.Length);
                  writer.WriteLine();
                  writer.Flush();
                  Stream output = writer.BaseStream;
                  Stream input = f.OpenRead();
                  byte[] buffer = new byte[4096];
                  for (int size = input.Read(buffer, 0, buffer.Length); size > 0; size = input.Read(buffer, 0, buffer.Length))
                  {
                      output.Write(buffer, 0, size);
                  }
                  output.Flush();
                  writer.WriteLine();
              }
              else
              {
                  writer.WriteLine(@"Content-Disposition: form-data; name=""{0}""", key);
                  writer.WriteLine();
                  writer.WriteLine(value.ToString());
              }
          }
      }

      public void WriteObject(Stream stream, object data)
      {
          StreamWriter writer = new StreamWriter(stream);
          if (data != null)
          {
              if (data is Dictionary<string, object>)
              {
                  foreach (var entry in data as Dictionary<string, object>)
                  {
                      WriteEntry(writer, entry.Key, entry.Value);
                  }
              }
              else
              {
                  foreach (var prop in data.GetType().GetFields())
                  {
                      foreach (var attribute in prop.GetCustomAttributes(true))
                      {
                          if (attribute is DataMemberAttribute)
                          {
                              DataMemberAttribute member = attribute as DataMemberAttribute;
                              WriteEntry(writer, member.Name ?? prop.Name, prop.GetValue(data));
                          }
                      }
                  }
                  foreach (var prop in data.GetType().GetProperties())
                  {
                      if (prop.CanRead)
                      {
                          foreach (var attribute in prop.GetCustomAttributes(true))
                          {
                              if (attribute is DataMemberAttribute)
                              {
                                  DataMemberAttribute member = attribute as DataMemberAttribute;
                                  WriteEntry(writer, member.Name ?? prop.Name, prop.GetValue(data, null));
                              }
                          }
                      }
                  }
              }
          }
          writer.Write("--");
          writer.Write(boundary);
          writer.WriteLine("--");
          writer.Flush();
      }
  }

The usage is as follows:
First a PHP file

<?php
print_r($_REQUEST);
$src = $_FILES['y']['tmp_name'];
$dest = "C:\\Windows\\Temp\\".$_FILES['y']['name'];
echo $src;
echo "\r\n";
echo $dest;
echo @copy($src, $dest);
?>

Then the Page control

    public partial class Page : UserControl
  {
      public Page()
      {
          InitializeComponent();
          // Create a request object
          HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri("http://localhost/rms/test.php"));
          OpenFileDialog dlg = new OpenFileDialog();
          if (dlg.ShowDialog().Value)
          {
              request.PostMultiPartAsync(new Dictionary<string, object> { { "x", "1" }, { "y", dlg.File } }, new AsyncCallback(asyncResult =>
              {
                  HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asyncResult);

                  Stream responseStream = response.GetResponseStream();
                  StreamReader reader = new StreamReader(responseStream);
                  this.Dispatcher.BeginInvoke(delegate
                  {
                      // output is a TextBlock
                      output.Text = reader.ReadToEnd();
                      response.Close();
                  });
              }));
          }
      }
  }

Since it is able to serialize data contract, you could actually replace

new Dictionary<string, object> { { "x", "1" }, { "y", dlg.File } }

with

new Point(){X=1, Y=2}

given the point class is like this:

    [DataContract]
  public class Point
  {
      [DataMember]
      public int X { get; set; }
      [DataMember(Name="y")]
      public int Y { get; set; }
  }

Slow Performance of PHP in Windows IIS

There are a few reasons that PHP runs slowly in Windows. Not because of the request delegation from IIS to PHP via FastCGI or CGI. In fact, the delegation is quite robust in FastCGI.

TCP/IP connection problem between database and web server may be a reason. Some Windows specific behaviors may be another. One of the Windows specific behaviors is the invocation of gethostbyaddr() function.

Here is the high level behavior of gethostbyaddr() function making the invocation slow:
Original

gethostbyaddr() on windows waits for WINS resolution timeout even when you disable WINS on the server.

This means that your DNS server may return "No hostname found" and windows gethostbyaddr() will just sit there, never having asked a WINS server anything, for 3 - 4 seconds. I watched it do just that with a sniffer.

Here is the root cause:
Original

Difference between gethostbyaddr and getnameinfo.

Originally posted by: Gregor The Eye

First at all, sorry for my english. It's not my native language, and, unfortunately, i havn't time to verify all text i enter.

Now, to subject. Computer may have many "names", and two main name groups of names computers have is a "DNS" names and a "NetBIOS" names.

The "NetBIOS" name is aname you assign to your computer using system->network identification->network ID. This name
is displayed in you "network neighborhood" and generally used in local networks to identify computers.

The "DNS" name is a name given to machine in internet. This name is a synonim to machine IP (becouse IP is\s not huma-friendly). "DNS" name is stored in special internet services (so-called "DNS" servers), not at the computer itself. Generally, computer can be shut off, but it's DNS name will be available and produce correct IP.

For example, if I install OS to new computer and call it "EYE", it will have a NetBIOS name "EYE", and, connecting it to my local network, i can access it in "network neighborhood". But, from the internet, i can't access to it using "EYE" name, and instead i must use it's IP - for example - 113.54.25.14. But if i need over users to access it from internet using a human-friendly name, i pay amount of money to sertain organization, and they register a name for it - for example - "www.GregorTheEye.com". Then, user can access my machine using this name, fopr example, writing:

ping www.GregorTheEye.com

or

telnet www.GregorTheEye.com

So, we have 2 names for machine - NetBIOS name and DNS name.
In my local network, following commands will be valid:

ping 113.54.25.14
ping EYE
ping www.GregorTheEye.com

Third command will be valid ONLY if my computer is conected to internet.

And from the internet, valid commands are:

ping 113.54.14
ping www.GregorTheEye.com

As you can see, NetBIOS name is not available from internet.

The main difference between NetBIOS name and DNS name is that DNS name will be available only if computer is connected to the internet and has i's name registered on it. NetBIOS name will be always available to computers that directly connected to target one.

To get DNS name, you must send a request to DNS server (it's IP is written in the system registry if you computer is connected to internet). If DNS server is unavailable, it will take default timeout time to discover this. If it's available, DNS server will return you human-friendly name of target machine if it's exist in database.

To obtain a NetBIOS name, you must send UDP packe to target machine and wait for responce. Becouse UDP is not a guaranteed-to-deliver protocol, responce may ot came, came corrupted, came many times etc. Your program must patiently wait for responce (it's not from 137 port - you must SEND UDP packet to 137 port of target machine, but responce will be returned to the port you specify. Only Win95 no OSR2 has a bug there responce always returns to port 137).

And, finally to Win32 functions -

gethostbyaddr() will first try to connect DNS server, and, if it's unavailable, will try to get NetBIOS name. So, it will get machine name one method or another in most cases.

getnameinfo() will ONLY try a DNS name, no attempts to get a NetBIOS name(). That's the difference.

If you write a program that must get names of local machines as well as names of internet machines (for example, network or port scanner), you MUST use gethostbyaddr(). If you write a program that will operate only with names of remote machines (IRC client, for example) - you MUST use getnameinfo() becouse Microsoft instructs so in lates MSDN releases.

If you have any questions - feel free to contact me. Also, if i write something wrony, i will be very thankfull if you send ma a hint.

Best regards, Gregor The Eye.

However, there is no getnameinfo() function in PHP. One of the nice feature in PHP documentation is the commenting. A commenter contributed a gethostbyaddr_timeout() function that only query the DNS server. You need to pass the DNS address (instead of using the OS specified DNS) to the function, though.

Flash vs. Silverlight vs. JavaFX

In RIA, Flash is the dominant player while Silverlight and JavaFX are new. However, Silverlight is in a better position than JavaFX.

.Net is a huge platform like Java. However, Microsoft trimmed .Net into a 5MB version to become a Silverlight runtime. On the other hand, Sun only added scripting syntax to Java and reuse the whole Java runtime as well as the branding. In fact, Java applet and Java web start has become the burden of the Java family. Users perceive Java is slow, insecure, and has compatibility issues whether or not any of them are true. So, it is no longer a technology issue. It is a marketing issue.

In fact, when it comes to marketing, Microsoft does it far much better than everyone one else in the technology industry.

Improving ASP.NET Web Performance with Local SMTP Server

Implementing public web site, we usually need an SMTP server for registration verification, reset password notification, etc. We usually utilize existing email infrastructure to send the email. However, we may not have the control of the infrastructure. The SMTP server may locate overseas or may go down sometime. In these cases, the response of the web server seems slow. We can setup the SMTP in local server to resolve the problem.

How does SMTP works?
There is a special record in DNS called MX record (Mail eXchange record). Using a@b.com as an example. The SMTP server inquires the MX record of b.com to resolve the SMTP server that will receive the email.

SMTP Setup in Windows
Windows comes with SMTP service in XP, 2003, Vista, and 2008. It can be configured with the following steps
1. In IIS Manage, right click Default SMTP Virtual Server, choose Properties
2. Choose Access tab and click the Relay button at the bottom in Relay restriction section. An SMTP relay lets an SMTP client use an SMTP server to forward e-mail messages to a remote domain. By default, it is disabled so you cannot send any email.
3. Add your local host address to allow your local application to send email. You can even add your internal network address so your other machine can use this as the SMTP relay.
4. Optionally, you can prohibit the external connection by configuring the Connection control section.

Since SMTP will queue the email and return, it responses instantly. The web performance also improves instantly, too.

HTML Robustness and Query String Parameter Naming

All the way back to 10 years ago, HTML was designed for non-professionals to publish information to the world wide web. The parsing and interpreting rules are loose. Web browsers implemented a lot of heuristics to display as much information as possible.

Now, we have been entered the age of Web 2.0 for a few years. The legacy loosen rules remains. One of the rules I have discovered today is the HTML entity. HTML entity without semicolon will be interpreted as if it has semicolon. For example, &lt will be interpreted as <.

The problem is when writing a web application. You will need to concatenate query string in <a> tag like <a href="list.php?a=x&b=y">. The query string parameters cannot be named as one of those HTML entities. So, <a href="list.php?a=x&lt=y"> does not work. The URL will be interpreted as list.php?a=x<=y. Certainly, no one would use lt as the name of a query string parameter. However, some other common names including euro, copy, pound, cent, uml, not, micro, times and divide will not work.

Developer Friendliness

People usually said that Firefox is more developer friendly than IE or Safari. Currently, there are two embarrassing incidents that flip my perception of Firefox.

1. Text box inside DIV
For some weird reason in Firefox 2, the cursor caret in the text box disappears when the text box is inside a DIV. Web developer has to work with the CSS hack to get the caret back

overflow: hidden; position:fixed;

It is OK if web developer have the total control of the HTML. However, we have web components in .Net and Java Server Face. If such components do not aware of this, user may had a bad experience.

2. Length Required with POST
Content-length is an HTTP header letting the server know how much data it should be expecting in the request. However, Firefox does not send such header to the server with posting with XMLHTTPRequest. Developer need to write it out explicitly

xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded')
xhr.setRequestHeader('Content-length', '0');

XMLHTTPRequest is the key doing AJAX! The HTTP request is usually uncachable when using POST. Why on earth developers need to code it themselves?

I think implementing the standard is important. It relieves the web developers headache. On the other hand, please do not introduce new problem to the developers.

ShineDraw in JavaScript: Kaleidoscope

This is the JavaScript version of Kaleidoscope in ShineDraw. It uses <canvas> and works with Gecko (Firefox, SeaMonkey), Webkit (Google Chrome, Safari) engines.

Performance:

• SeaMonkey - Comsumes a lot of CPU power
• Google Chrome - Renders the Kaleidoscope smoothly.

ShineDraw in JavaScript: Random Expand Rotator

This is the JavaScript version of Random Expand Rotator in ShineDraw.


Implementation Note:
Image scaling and clipping cannot be done at the same time in JavaScript.

ShineDraw in JavaScript: Waving Text

This is the JavaScript version of Waving Text in ShineDraw.

How Bad Can a Bad MPEG Decoder Be?

I have be using VirtualDub for years to encode video files. I found many criticisms VirtualDub is being old. We should use more modern encoder support new file format like MP4. Avidemux seems very popular as it has as GUI. No command line option to be remembered.

I used Avidemux to try to encode a few files and it seems that it could get the job done. Until recently, I found a very unforgivable feature (a.k.a. a big bad bug)! The frame detail got blurred! I thought it was a blurred by the codec at the beginning. However, when I look at the preview screen in Avidemux, the details has been destroyed already comparing with VirtualDub and Media Player Classic (which deinterlaced the frame and still looks good).

We want to minimize the quality loss of the video in many ways like increasing bitrate. Losing the quality while decoding the source file is not acceptable. Please trash it right the way!

Taking Google Maps API Offline

Sometimes, we want to utilize the some API for other work. Google Maps API is a very good example. You may want to use it as a photo viewer. Like allowing user to zoom-in and zoom-out, tagging, etc. However, since the API needs to validate the key online, you cannot take it as a standalone application where people can use offline.

Now, Google Maps API is a JavaScript API which we can download the script from the web. It just needs a bit hacking to get the API offline:
1. Highly obfuscated code
2. Dynamic module loading
3. Key verification

The obfuscated code can be ignored since you do not need to understand everything. All you need is finding the key statements that involve loading resources online (like module loading, image loading, etc). You can use a formatter to beautify it a little bit and decode it mentally.

The key verification is the most easiest part. Like all hacking, find the verification function and return a true will be good.

The process can be done within an hour for an experienced programmer.

With a good packager, we can have a really nice image mosaic viewer offline.

Microsoft Servers eBooks

There has been some times that we want to play with some server but have no idea how to start with. Here are some of the Microsoft Servers eBooks for installation.

• SQL Server 2005 Books Online (September 2007)
• BizTalk Server 2006 R2 Installation and Upgrade Guides
• Installation Guide for Microsoft Office SharePoint Server 2007
• Deploying a Standard Exchange Server 2007 Organization
• Microsoft Dynamics CRM 4.0 Implementation Guide
• Deployment Guide for PerformancePoint Server 2007

Receiving AS2 MDN in BizTalk Orchestration

Having an AS2 port setup in an orchestration is easy to configure a one-way send port. However, configure a request-response send-port is a bit tricky. You probably know the message schema of the send port but you have no idea of the schema of an MDN.

You may attempt to use System.Object, System.String, or even create an MIME message type with two parts. However, none of these options work. Going through the schemas under the BizTalk EDI Application in Administration Console, you only see the property schema of EdiIntAs.EdiIntProperties and no sign for other schema for MDN.

At this moment, you will want to have a type that receiving any message. Microsoft.XLANGs.BaseTypes.Any is the savior. You can choose it from Schema > Select from referenced assembly...

Moving to .Net 3.0/3.5

WF, WCF, WPF, and LINQ are going to be pushed by Microsoft. To be honest, the framework is just unbeatable by other current programming frameworks. Adobe has the equivalent technologies but they are more on documents only and have not fully integrated together yet. Open source camp is too scattered. It is hard to imagine it can response quickly to the .Net 3.5 wave.

A good artical about C# and VB.Net

Despite that the article has been written for a long time, it is still a good article to read.
http://www.codeproject.com/dotnet/CSharpVersusVB.asp

Thoughts on Silverlight

Microsoft is pushing Silverlight so hard these days. As the official release is approaching, the adoption rate will be low if it use the same installation method of beta. A better off will be Flash like installation. Everything can be installed automatically and no restart of browser. The real frictionless environment.

I could provision it could be a big success when Microsoft solves this particular problem.