Tuesday, July 14, 2015

Building a Source Code Generator with AngularJS

If you are looking for generating scaffold for AngularJS, this is not what this blog post about. This blog post is about generate source code with AngularJS.

The principle of creating source code generators is pretty simple (details here): a model, a template, and a driver. Given:
  1. HTML5 can access to local files as the data model
  2. AngularJS has its template language
  3. JavaScript is the driver
The following code in JSFiddle takes a tab delimited (3 columns) file to generate a Oracle script:


The syntax highlighting is done by SHJS.

Since it takes any JavaScript object as the model, other potential input formats are JSON and XML (with aid of some library). This is a lightweight cross-platform source code generator.

Friday, May 23, 2014

BizTalk Atomic Update for Side-by-Side Orchestration Deployment

Deploying orchestration side-by-side, you have to unenlist the old version and start the new version. To minimize the down time, the process should be done in atomic way so that the old orchestration would not pick the new messages while the new orchestration instances will be activated by the new messages.

BizTalk exposes the .Net API to manage the orchestrations. The following script demonstrates how to use PowerShell to invoke the API so that unenlisting and starting the orchestrations could be done atomically.

To use the script, update $connectionString and the expressions in $unenlisting and $starting variables so that they could match the assembly qualified names of the orchestrations. The script also takes one argument "test" or "trial" so that you could test out whether the expressions are correct.
### Settings of the script ###
$test = $FALSE
If ($args.Count -gt 0) {
    $test = ($args[0] -eq "test") -or ($args[0] -eq "trial")
}
$connectionString = "SERVER=.;DATABASE=BizTalkMgmtDb;Integrated Security=SSPI"
$unenlisting = `
    @( `
      '^Test.*' `
    )
$starting = `
    @( `
      '.*SubOrchestration.*' `
    )

### Initialize BizTalk catalog explorer ###
add-type -assemblyName "Microsoft.BizTalk.ExplorerOM, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL"
$catalog = New-Object -TypeName Microsoft.BizTalk.ExplorerOM.BtsCatalogExplorer
$catalog.ConnectionString = $connectionString

### Matching orchestration's assembly qualified name with the regular expressions in $unenlisting and $starting array ###
If ($test) {
    Write-Host "TEST MODE" -foregroundcolor white
    Write-Host
}
Try
{
    $catalog.Applications | ForEach-Object {
        $_.Orchestrations | ForEach-Object {
            $target = $_
### -like is the wildcard operator
### -match is the regular expression operator

#            $unenlisting | Where-Object {($target.AssemblyQualifiedName -like $_) -and ($target.Status -ne [Microsoft.BizTalk.ExplorerOM.OrchestrationStatus]::Unenlisted)} | ForEach-Object {
            $unenlisting | Where-Object {($target.AssemblyQualifiedName -match $_) -and ($target.Status -ne [Microsoft.BizTalk.ExplorerOM.OrchestrationStatus]::Unenlisted)} | ForEach-Object {
                Write-Host "Unenlisting" $target.FullName $target.BtsAssembly.Version "..." -foregroundcolor yellow
                $target.Status = [Microsoft.BizTalk.ExplorerOM.OrchestrationStatus]::Unenlisted
            }
#            $starting | Where-Object {($target.AssemblyQualifiedName -like $_) -and ($target.Status -ne [Microsoft.BizTalk.ExplorerOM.OrchestrationStatus]::Started)} | ForEach-Object {
            $starting | Where-Object {($target.AssemblyQualifiedName -match $_) -and ($target.Status -ne [Microsoft.BizTalk.ExplorerOM.OrchestrationStatus]::Started)} | ForEach-Object {
                Write-Host "Starting   " $target.FullName $target.BtsAssembly.Version "..." -foregroundcolor green
                $target.Status = [Microsoft.BizTalk.ExplorerOM.OrchestrationStatus]::Started
            }
        }
    }
    Write-Host
    If ($test) {
        $catalog.DiscardChanges()
        Write-Host "Changes rolled-back for test mode" -foregroundcolor white
    }
    Else {
        $catalog.SaveChanges()
        Write-Host "Changes committed" -foregroundcolor white
    }
}
Catch {
    $catalog.DiscardChanges()
    Write-Host "Changes rolled-back" -foregroundcolor red
    Write-Host $Error cyan
}
Write-Host 

Wednesday, March 27, 2013

Troubleshooting Slow BizTalk Admin Console

Since SQL Server is the backend storage of BizTalk, performance of SQL Server affects the rest of the BizTalk components. When the BizTalk Admin Console is running slowly
  • Check the sizes of BizTalk databases
  • Check the connectivity to the SQL Server
    • Open a connection with SQL Server Management Studio see if it behaves the same
    • Check for network connectivity
    • Try out different protocol configurations
      • Operation guys may comes in and update the SQL's port number to something rather than the default 1433 port leaving your BizTalk server untouched. The SQL client on the BizTalk server falls back to use Named Pipe after the TCP timeout (depends on the priority of protocol).
  • Try set the Recovery Mode of all BizTalk databases from Full to Simple
    • Not recommended in production environment
    • You should know the implication. If you do not, please read MSDN.

Saturday, June 16, 2012

Time-outs

IIS Management Console

  • Application Pool
    • Advanced Settings
      • Idle Time-out
        One way to conserve system resources is to configure idle time-out settings for the worker processes in an application pool. When these settings are configured, a worker process will shut down after a specified period of inactivity. The default value for idle time-out is 20 minutes.

  • Web Site
    • Advanced Settings
      • Connection Limits
        • Connection Time-out
          Specifies the time (in seconds) that IIS waits before it disconnects a connection that is considered inactive. Connections can be considered inactive for the following reasons:
          • The HTTP.sys Timer_ConnectionIdle timer expired. The connection expired and remains idle.
          • The HTTP.sys Timer_EntityBody timer expired. The connection expired before the request entity body arrived. When it is clear that a request has an entity body, the HTTP API turns on the Timer_EntityBody timer. Initially, the limit of this timer is set to the connectionTimeout value. Each time another data indication is received on this request, the HTTP API resets the timer to give the connection more minutes as specified in the connectionTimeout attribute.
          • The HTTP.sys Timer_AppPool timer expired. The connection expired because a request waited too long in an application pool queue for a server application to dequeue and process it. This time-out duration is connectionTimeout.
          The default value is 00:02:00 (two minutes).
    • Features View
      • Configuration Editor
        • Section = system.web/httpRuntime
          • executionTimeout
            Specifies the maximum number of seconds that a request is allowed to execute before being automatically shut down by ASP.NET.

            This time-out applies only if the debug attribute in the compilation element is False. Therefore, if the debug attribute is True, you do not have to set this attribute to a large value in order to avoid application shutdown while you are debugging.

            The default is 110 seconds.

Service Configuration Editor (system.serviceModel)

  • Bindings 
    • closeTimeout
      A TimeSpan value that specifies the interval of time provided for a close operation to complete. This value should be greater than or equal to Zero. The default is 00:01:00.
    • openTimeout

      A TimeSpan value that specifies the interval of time provided for an open operation to complete. This value should be greater than or equal to Zero. The default is 00:01:00.
    • receiveTimeout
      A TimeSpan value that specifies the interval of time provided for a receive operation to complete. This value should be greater than or equal to Zero. The default is 00:10:00.
    • sendTimeout
      A TimeSpan value that specifies the interval of time provided for a send operation to complete. This value should be greater than or equal to Zero. The default is 00:01:00.

 BizTalk Server Administration Console

  • BizTalk Group
    •  Platform Settings
      • Hosts
        • Settings
          • General
            • Response Time [see also]
              Specify the default timeout for request response messages.
              When an adapter submits a request-request message, it needs to specify the time-out of the request message on the IBTTransportBatch.SubmitRequestMessage Method (COM) API. A response message will be delivered to the adapter only within this time-out period. After the time-out expires, a negative acknowledgment (NACK) will be delivered to the adapter instead of the response message. If the adapter does not specify a time-out value, the engine uses the default value of 20 minutes.
      • Applications
        • [Application]
          • Send Ports/Receive Locations
            • [Send Port/Receive Location]
              • Properties
                • Type = Any WCF Adapter > Configure...
                  • Binding

    Wednesday, April 27, 2011

    ASP.NET MVC View Engine with FreeMarker

    FreeMarker is a quite success template engine in Java world. It is used with Apache Struts, an MVC framework, as a view engine. ASP.NET MVC is using Web Form as the default view engine. The problem is the view become a spaghetti of HTML and C# snippets fairly quickly, just like classic ASP and PHP. FreeMarker.Net is a project that ports the FreeMaker as an ASP.NET MVC view engine

    The idea is compiling FreeMarker to .Net assembly with IKVM and create a wrapper to wrap .Net objects so that FreeMarker can understand.

    Compiling FreeMarker
    It is a strength forward process. It can be done with one command:
    ikvmc freemarker.jar -target:library

    Separating Necessary IKVM Libraries
    Only the following libraries is required for development:
    • IKVM.OpenJDK.Beans.dll 
    • IKVM.OpenJDK.Charsets.dll
    • IKVM.OpenJDK.Core.dll
    • IKVM.OpenJDK.SwingAWT.dll
    • IKVM.OpenJDK.Text.dll
    • IKVM.OpenJDK.Util.dll
    • IKVM.Runtime.dll

    Wrapping .Net Object
    FreeMarker does not directly deal with the objects. Instead, it deals with the TemplateModel objects. There are a few template models to be implemented:
    • TemplateBooleanModel
    • TemplateDateModel
    • TemplateHashModel
    • TemplateMethodModelEx
    • TemplateNumberModel
    • TemplateScalarModel
    • TemplateSequenceModel
    FreeMarker provides an ObjectWrapper interface to wrap the raw objects into TemplateModel.

    The NetObjectModel is actually is a TemplateHashModel
    public class NetObjectModel : StringModel, 
                                  TemplateModel, 
                                  TemplateHashModel {
            Dictionary<string, PropertyInfo> props = 
                new Dictionary<string, PropertyInfo>();
            Dictionary<string, MethodModel> methods = 
                new Dictionary<string, MethodModel>();
            Dictionary<string, ExtensionMethodModel> extensionMethods = 
                new Dictionary<string, ExtensionMethodModel>();
    
            public NetObjectModel(object data, 
                                  NetObjectWrapper wrapper)
                : base(data, wrapper){
                var type = data.GetType();
                foreach (var p in type.GetProperties()) {
                    props.Add(p.Name, p);
                }
                foreach (var m in type.GetMethods()) {
                    if (!methods.ContainsKey(m.Name)) {
                        methods.Add(m.Name, 
                                    new MethodModel(data,  
                                                    wrapper, 
                                                    m.Name));
                    }
                }
            }
    
            public virtual TemplateModel get(string key) {
                if (props.ContainsKey(key)) {
                    return wrapper.wrap(props[key].GetGetMethod()
                                                  .Invoke(data, null));
                }
                else if (methods.ContainsKey(key)) {
                    return methods[key];
                }
                else if (wrapper.ExtensionTypes.Count > 0) {
                    if (!extensionMethods.ContainsKey(key)) {
                        extensionMethods[key] = 
                            new ExtensionMethodModel(data, 
                                                     wrapper, 
                                                     key);
                    }
                    return extensionMethods[key];
                }
                else {
                    return TemplateModel.__Fields.NOTHING;
                }            
            }
    
            public virtual bool isEmpty() {
                return props.Count == 0;
            }
        }
    

    To adapt the ASP.NET objects, three more template model are created:
    • HttpApplicationStateModel
    • HttpRequestModel
    • HttpSessionStateModel
    They are similar but not in the same interface, all of them are like this:
    public class HttpApplicationStateModel : NetObjectModel, 
                                             TemplateModel, 
                                             TemplateHashModel {
    
            public HttpApplicationStateModel(
                 object data, 
                 NetObjectWrapper wrapper)
                : base(data, wrapper) {
                
            }
    
            public override TemplateModel get(string key) {
                HttpApplicationStateBase dic = 
                    data as HttpApplicationStateBase;
                if (dic.Keys.Cast<string>().Contains(key)) {
                    return wrapper.wrap(dic[key]);
                }
                else {
                    return base.get(key);
                }
            }
    
            public override bool isEmpty() {
                IDictionary<string, object> dic = 
                    data as IDictionary<string, object>;
                return dic.Count == 0 && base.isEmpty();
            }
        }
    View Engine
    The view engine is fairly simple, it simply initialize the FreeMarker configuration.
    public class FreemarkerViewEngine : VirtualPathProviderViewEngine {
            Configuration config;
            public FreemarkerViewEngine(string rootPath, 
                                        string encoding = "utf-8") {
                config = new Configuration();
                config.setDirectoryForTemplateLoading(
                    new java.io.File(rootPath));
                AspNetObjectWrapper wrapper = 
                    new AspNetObjectWrapper();
    
                config.setObjectWrapper(wrapper); 
                base.ViewLocationFormats = 
                    new string[] { "~/Views/{1}/{0}.ftl" };
                config.setDefaultEncoding(encoding);
                base.PartialViewLocationFormats = 
                    base.ViewLocationFormats;
            }
    
            protected override IView CreatePartialView(
                ControllerContext controllerContext, 
                string partialPath) {
                return new FreemarkerView(config, partialPath);
            }
    
            protected override IView CreateView(
                ControllerContext controllerContext, 
                string viewPath, 
                string masterPath) {
                return new FreemarkerView(config, viewPath);
            }
        }
    

    View
    FreemarkerView implements IView to render the content. It simply create the dictionary as variable bindings and invoke the FreeMarker.
    public class FreemarkerView : IView{
        public class ViewDataContainer : IViewDataContainer {
            private ViewContext context;
            public ViewDataContainer(ViewContext context) {
                this.context = context;
            }
    
            public ViewDataDictionary ViewData {
                get {
                    return context.ViewData;
                }
                set {
                    context.ViewData = value;
                }
            }
        }
    
        private freemarker.template.Configuration config;
        private string viewPath;
        public FreemarkerView(freemarker.template.Configuration config, string viewPath) {
            this.config = config;
            this.viewPath = viewPath;
        }
    
        public void Render(ViewContext viewContext, 
                            System.IO.TextWriter writer) {
            Template temp = config.getTemplate(viewPath.Substring(2));
                
            Dictionary<string, object> data = 
                            new Dictionary<string, object>{
                {"model", viewContext.ViewData.Model},
                {"session", viewContext.HttpContext.Session},
                {"http", viewContext.HttpContext},
                {"request", viewContext.HttpContext.Request},
                {"application", viewContext.HttpContext.Application},
                {"view", viewContext},
                {"controller", viewContext.Controller},
                {"url", new UrlHelper(viewContext.RequestContext)},
                {"html", new HtmlHelper(viewContext, 
                                new ViewDataContainer(viewContext))},
                {"ajax", new AjaxHelper(viewContext, 
                                new ViewDataContainer(viewContext))},
            };
    
            Writer output = new JavaTextWriter(writer);
            temp.process(data, output);
            output.flush();
        }
    }
    
    Configuration
    Configuration is as simple as adding the view engine to the view engine collections.
    protected void Application_Start() {
        AreaRegistration.RegisterAllAreas();
    
        // ****** Optionally, you can remove all other view engines ******
        //ViewEngines.Engines.Clear();
        ViewEngines.Engines.Add(new FreemarkerViewEngine(this.Server.MapPath("~/")));
    
        RegisterGlobalFilters(GlobalFilters.Filters);
        RegisterRoutes(RouteTable.Routes);
    }
    

    Extension Methods
    ASP.NET MVC relies on extension method quite heavily. In the NetObjectWrapper, the following code is added to find the extension methods:
    public virtual void AddExtensionNamespace(string ns) {
        var types = this.ExtensionTypes;
        foreach (var a in AppDomain.CurrentDomain.GetAssemblies()) {
            try {
                foreach (var t in a.GetExportedTypes()) {
                    if (!types.Contains(t) && 
                        t.IsClass && 
                        t.Name.EndsWith("Extensions") && 
                        t.Namespace == ns && 
                        t.GetConstructors().Length == 0) {
                        types.Add(t);
                    }
                }
            }
            catch { }
        }
    }
    
    In the view engine's constructor, it reads the namespaces specified under system.web/pages sections in web.config:
    PagesSection section = 
        (PagesSection)WebConfigurationManager
                          .OpenWebConfiguration("~/")
                          .GetSection("system.web/pages");
    if (section != null) {
        foreach (NamespaceInfo info in section.Namespaces) {
            wrapper.AddExtensionNamespace(info.Namespace);
        }
    }
    
    An ExtensionMethodModel class is created to find the appropriate method to invoke:
    foreach (var type in wrapper.ExtensionTypes) {
        MethodInfo method = type.GetMethod(
            methodName, 
            BindingFlags.Public | BindingFlags.Static, 
            Type.DefaultBinder, 
            argTypes, null);
        if (method != null) {
            cache.Add(key, method);
            return wrapper.wrap(method.Invoke(target, args.ToArray()));
        }
    }
    
    Localization
    ResourceManagerDirectiveModel (a TemplateDirectiveModel) and ResourceManagerModel are created so that we could do something like this:
    <@resource type="Freemarker.Net.MvcWeb.App_GlobalResources.PersonResource, Freemarker.Net.MvcWeb"/>
    

    ${res.Title}

    ResourceManagerDirectiveModel is getting the ResourceManager from the resource and put it into res template variable:
    public void execute(freemarker.core.Environment env, java.util.Map parameters, TemplateModel[] models, TemplateDirectiveBody body) {
        if (parameters.containsKey("type")) {
            TemplateScalarModel scalar = 
                (TemplateScalarModel)parameters.get("type");
            var type = Type.GetType(scalar.getAsString());
            env.setVariable("res", 
                env.getObjectWrapper()
                    .wrap(type.GetProperty("ResourceManager", 
                                            BindingFlags.Static | 
                                            BindingFlags.NonPublic | 
                                            BindingFlags.Public)
                            .GetGetMethod(true)
                            .Invoke(null, null)));
        }            
    }
    
    Adding More Directives
    To all more directives can be added in the future, MEF is used. The directive implementation only needs to export and implement the ImportableDirective interface.
    [Export(typeof(ImportableDirective))]
        public class ResourceManagerDirectiveModel : TemplateDirectiveModel, ImportableDirective {
    
    The view engine will import them in the constructor:
    public class FreemarkerViewEngine : VirtualPathProviderViewEngine {
            Configuration config;
            [ImportMany]
            IEnumerable<ImportableDirective> directives;
            public FreemarkerViewEngine(string rootPath, 
                                        string encoding = "utf-8") {
                // initialize the config 
                // ...
                // import the extension methods' namespaces
                // ...
                // import the directives
                var dir = new DirectoryInfo(
                    Path.Combine(rootPath, "bin"));
                var catalogs = dir.GetFiles("*.dll")
                    .Where(o => o.Name != "freemarker.dll" && 
                               !(o.Name.StartsWith("IKVM.") || 
                               o.Name.StartsWith("Freemarker.Net")))
                    .Select(o => new AssemblyCatalog(
                                 Assembly.LoadFile(o.FullName))
                );
                var container = new CompositionContainer(
                    new AggregateCatalog(
                        new AggregateCatalog(catalogs),
                        new AssemblyCatalog(
                          typeof(ImportableDirective).Assembly)
                ));
                container.ComposeParts(this);
            }
    

    The source code is available in CodePlex. Please visit http://freemarkernet.codeplex.com/.

    P.S.: The bonus of this project is we have not only a new view engine in ASP.NET MVC, but also a new template engine in .Net Framework. Maybe someday we could use FreeMarker to generate code instead of T4 in Visual Studio SDK.

    Thursday, April 21, 2011

    Caching in .Net Framework 4

    We used to have caching in ASP.NET. For non-web application caching, Cache Application Block from Enterprise Library may be the choice. In .Net Framework 4, caching is baked into the library and no longer limited to ASP.NET.

    After adding System.Runtime.Caching assembly as the reference in your project, we can cache the data retrieved by a web services:
    using System.Runtime.Caching;
    class CrmHelper{
        static ObjectCache cache = MemoryCache.Default;
    //...
        string key = string.Format("{0}/{1}", 
                              "account", "address1_addresstypecode");
        MetadataService service = new MetadataService();
        service.Credentials = CredentialCache.DefaultCredentials;
    
        Option[] options = null;
        if (cache.Contains(key)){
            options = (Option[])cache[key];
        else {
            var picklist = (PicklistAttributeMetadata)
                service.RetrieveAttributeMetadata(
                    "account", "address1_addresstypecode");
            options = picklist.Options;
            cache.Set(key, options, new CacheItemPolicy {
                AbsoluteExpiration = DateTimeOffset.Now.AddMinutes(2)
            });
        }
    //...
    }
    if (cache.Contains(key)) ... cache.Set(key, data) is going to be a repeating pattern. We could write an extension method to wrap it up:
    public static class Extensions {
        public static T Get<T>(this ObjectCache cache, 
                               string key,
                               Func<T> retrieveFunction) {
            if (cache.Contains(key)) {
                return (T)cache[key];
            }
            else {
                var result = retrieveFunction();
                cache.Set(key, result, new CacheItemPolicy {
                    AbsoluteExpiration = 
                        DateTimeOffset.Now.AddMinutes(2)
                });
                return result;
            }
        }
    }
    
    The previous code is now cleaned up like this:
    using System.Runtime.Caching;
    class CrmHelper{
        static ObjectCache cache = MemoryCache.Default;
    //...
        string key = string.Format("{0}/{1}", 
                              "account", "address1_addresstypecode");
        MetadataService service = new MetadataService();
        service.Credentials = CredentialCache.DefaultCredentials; 
    
        Option[] options = cache.Get(key, () => {
            var picklist = (PicklistAttributeMetadata)
                service.RetrieveAttributeMetadata(
                    "account", "address1_addresstypecode");
            return picklist.Options;
        });
    //...
    }
    
    It is worth to note that the Cache Application Block from Enterprise Library is deprecating (http://msdn.microsoft.com/en-us/library/ff664753(v=PandP.50).aspx):
    Caching Application Block functionality is built into .NET Framework 4.0; therefore the Enterprise Library Caching Application Block will be deprecated in releases after 5.0. You should consider using the .NET 4.0 System.Runtime.Caching classes instead of the Caching Application Block in future development.
    If you only want a quick memory cache, the new caching API might already fit your needs. However, Enterprise Library provides some other implementations of caching (e.g. database, file, etc). If you want something more fancy, Enterprise Library could be an alternative.

    Refereces:

    Sunday, March 27, 2011

    WCF Channel Factory and Unity 2.0

    Unity 2.0 does not support ChannelFactory in configuration natively. However, we could extend Unity. Inspired by Chris Tavares's FactoryElement extension for static factory configuration, here is the ChannelFactory extension for WCF channel factory configuration in Unity.

    The element will get the generic type of the ChannelFactory and calling the CreateChannel() method upon the injection.
    public class ChannelFactoryElement : InjectionMemberElement {
            private const string endpointConfigurationNamePropertyName = "endpointConfigurationName";
            private static int numFactories;
            private readonly int factoryNum;
    
            public ChannelFactoryElement() {
                factoryNum = Interlocked.Increment(ref numFactories);
            }
    
            [ConfigurationProperty(endpointConfigurationNamePropertyName, IsKey = false, IsRequired = true)]
            public string Name {
                get { return (string)base[endpointConfigurationNamePropertyName]; }
                set { base[endpointConfigurationNamePropertyName] = value; }
            }
    
            public override string Key {
                get { return "ChannelFactory " + factoryNum; }
            }
    
            public override IEnumerable<injectionmember> GetInjectionMembers(IUnityContainer container, Type fromType,
                Type toType, string name) {
    
                Type factoryType = typeof(ChannelFactory<>).MakeGenericType(toType);            
                var constructor = factoryType.GetConstructor(new Type[] { typeof(string) });
                var factory = constructor.Invoke(new object[]{Name});
                var method = factoryType.GetMethod("CreateChannel", Type.EmptyTypes);
                return new InjectionMember[] { new InjectionFactory(o => method.Invoke(factory, null)) };
            }
    
        }
    

    The extension will register the factory as the element name of ChannelFactoryElement.
    public class ChannelFactoryConfigExtension : SectionExtension {
            public override void AddExtensions(SectionExtensionContext context) {
                context.AddElement<ChannelFactoryElement>("factory");
            }
        }
    

    The factory element uses endpointConfigurationName to retrieve the client's endpoint as in ChannelFactory(endpointConfigurationName) constructor.
    <unity xmlns="http://schemas.microsoft.com/practices/2010/unity">
        <sectionExtension type="MyNamespace.ChannelFactoryConfigExtension, MyAssembly" />
        <container>      
          <register type="MyNamespace.IMyService, MyAssembly">
            <factory endpointConfigurationName="testing"/>
          </register>
        </container>
      </unity>