Sunday, July 6, 2014

Dapper 

Dapper is an object-relational mapping (ORM) solution for the Microsoft .NET platform. It is written for and used by stackoverflow. It has the signatures of a stackoverflow baby, hugely popular, no-bullshit/lightweight, effective and efficient, stellar performance.

We have been using Dapper for a couple of years. We love it, for its simplicity and performance. It is so simple, whether we do select, insert or update, whether we use stored procedures or ad-hoc sql.


using (var connection = new SqlConnection(myConnectionString))
            {
                connection.Open();
                const string sqlStatement = @"select client_id As Id, client_name As Name from dbo.client with(nolock) ";
                
                return connection.Query(sqlStatement);
            }

.NET async 

.net 4.5 came out about 2 years ago, and asynchronous programming with async and await is one of the big features. async/await allows you to define a Task that you can await on, and then continue execution once the work is done, all without blocking the calling thread unnecessarily. async and await operations took away the programmatic complexity from programmers, however, under the hood the compiler still does the heavy lifting of setting up a waiting context/state machine so a task can get back to exactly where it has begun.

 The overhead of async/await in NET 4.5 demonstrates that "Despite this async method being relatively simple, ANTS Performance Profiler shows that it’s caused over 900 framework methods to be run in order to initialize it and the work it does the first time that it’s run."


 It is agreed upon that async/await should be limited to only I/O heavy operations, web access, working with files, working with images.

How about database calls? Turned out database calls are by nature blocking. It blocks the calling thread even if you stamp your method as async and use the await keyword.

 To allow asynchronous database queries and other crud operation, dapper added a set of QueryAsync methods using .NET Framework 4.5's Task-Based Asynchronous Pattern. Under the hood, the asyn dapper methods uses connection.OpenAync, executeReaderAsync to allow connections to be opened and queried in asynchoronous mode.
      public static async Task> QueryAsync(this IDbConnection cnn, CommandDefinition command)
        {
            object param = command.Parameters;
            var identity = new Identity(command.CommandText, command.CommandType, cnn, typeof(T), param == null ? null : param.GetType(), null);
            var info = GetCacheInfo(identity, param);
            bool wasClosed = cnn.State == ConnectionState.Closed;
            using (var cmd = (DbCommand)command.SetupCommand(cnn, info.ParamReader))
            {
                try
                {
                    if (wasClosed) await ((DbConnection)cnn).OpenAsync().ConfigureAwait(false);
                    using (var reader = await cmd.ExecuteReaderAsync(command.CancellationToken).ConfigureAwait(false))
                    {
                        return ExecuteReader(reader, identity, info).ToList();
                    }
                }
                finally
                {
                    if (wasClosed) cnn.Close();
                }
            }

        }
Should you use asynchronous database calls?

 Just because you can do it does not mean you should do it, says RickAndMSFT at here.

 True.

 It does your more harm than good if you queryAsync a small query that takes a few millisecond, however for long running database calls you may consider using querySync feature to avoid bottleneck and gain some responsiveness.

 Check out some benchmark testing from this article Asynchronous Database Calls With Task-based Asynchronous Programming Model (TAP) in ASP.NET MVC 4.

Thursday, July 3, 2014

About 4 years ago, I got a job in a medium-sized company. Before that, I worked in a tiny company. Before that, I worked in a smaller than tiny organization (not for profit). Life in those tinier-to-tiny companies were less hurried, often stagnant.

I would say, life in my current company is fulfilling, at least, very, very, very busy. However, my near-four-year stay at the company has seen (many) people come and go or let go with the fluidity of a free market.

Free market it is.

Sometimes the company let people go in bad times and good times. In bad times, it wanted cut cost so it does not soak in red; In good times, it wanted to "better align company goals and resources" (whatever that means).


A lot of times, people just leave. Why do they leave? There could be bazillions of reasons, for better benefit, better management, better, cooler technologies on the other side of the meadow ... After all, we are not Google, even Google engineers leave.

(An funny anecdote, when I was interviewed, my interviewer enthusiastically told me: We are not Google. Google only want people from, like, Stanford. But we are the next best thing to Google. I was fascinated and held on to that dearly. The next thing to Google sounds just as good as google. Turned out, there is an infinite space between us and google.)

What the impact of people leaving? Especially when the core engineers left.

Culturally ...

I often encounters articles/books about how to be a good programmer, the traits of a good programmer, pragmatic programmers. See here, here, or just here. There all have a grain of truth in it, but I also know, a good programmer is not just born a good one, often, there is a culture that shapes, disciplines,  guides and goads him/her to become one. And the culture is nothing but a bunch of core engineers that hold the same values and principles.

To be honest, good engineers are valuable commodities that are hard to find. To be honest still, we are not google, which makes finding good ones incredibly hard. Hours and hours spent interviewing, sifting through candidates. Weeks and months training a newbie to get know your product,  architecture, coding standards, and the peculiar technologies you are unfortunately heavily dependent upon.

They may or may not become as good as you have hoped. They may excel, be better than you. But they may also leave.

Productivity-wise

Your productivity will take a huge hit, your performance will suffer. Because instead of the x people working together at an established pace, you are now only one and you have divide your energy into putting-out fire, coding, training people and wondering why people leave and who is next.

Ugh.

Too bad we are not Google. People are leaving, even at a time when the company is prosper, and clients like Google and Yahoo kept jumping in our fish net




Tuesday, July 1, 2014

Let's face it, designing a web template is quite some work, not to mention it takes an eye of aesthetics and the fact that it has to look good on both web and mobile devices. Yes, any web is declaring its own death if it looks crappy on mobile.  So unless you are a professional designer and swimming in loads of money, if you need a quick template, you probably just grab one out there and tweak it and style it to your heart's desire.

That is what I did. And that is why I am so delighted by the various templates at W3 layouts. It has a big collection, and FREE and RESPONSIVE. None of the other template sites come close. (Other popular ones are templates at http://html5up.net/, but frankly I think they are mostly ugly or perfunctory. WrapBootStrap has some nice ones, but they are not free, though most of them do not cost you most 4 cups of Starbucks.)

Peculiar enough, the templates at W3 layouts often come in three flavors: web, mobile, smart phone. It took me a while to find out that smart phones are for our regular mobile chops: IPhone, android, windows etc. And Mobile templates are designed to use wap 2.0 standards with xhtml where javaScript is not supported. Heck, I guess i can safely discard mobile templates.

Then, the nagging question (quotes from w3 layouts.com):

When we have responsive WEB template why we need separate SMARTPHONE and MOBILE template?

Responsive design is good for users having speed network connection (WIFI, 3G). Responsive design will load the whole HTML, CSS and images which are used for desktop design, users in slow data connection have to wait and spend lot of data, time and money to load page. To save users Money and load time we have to use separate design depending on device compatibility.


So I set out to create my little personal site with the fancy and pleasant designs of W3 layouts. Oops, seems that a big part of the world still has not convert to the Microsoft/Asp .net camp, sadly, seems that more and more developers are parting their ways from the gloriously clunky world of Microsoft. (Proof: Why I Left the .NET Framework).

So W3 layouts is in the php camp, and they use a bit of php code to detect user browsers/devices. I would love to use php, but with the time constraint, I am also happy to stick to asp.net.

So happily and quickly searching (since everything is out somewhere if you search!). I found the following nifty code to get just what I want (got to say it is much more neat and clever than the php code provided by w3 layouts.


C# code (from Mobile Device Detection in asp.net)


  public static bool IfBrowserIsMobile()
        {
            Debug.Assert(HttpContext.Current != null);

            if (HttpContext.Current.Request != null && HttpContext.Current.Request.ServerVariables["HTTP_USER_AGENT"] != null)
            {
                var u = HttpContext.Current.Request.ServerVariables["HTTP_USER_AGENT"].ToString();

                if(u.Length < 4)
                    return false;

                if (MobileCheck.IsMatch(u) || MobileVersionCheck.IsMatch(u.Substring(0, 4)))
                    return true;
            }

            return false;
        }


The following is the php counterpart:


Saturday, September 7, 2013

Last Friday, we have had a big production release (Yes, instead of incremental daily releases, we still work in a 6-week cycle, adding new features, intenstive QA, then release).

Almost immediately after the release, we had a small trickle of errors discovered. In all, there had been about 6 errors discovered by users or by our product managers. Every time, if I realized that the error was introduced by something I did, my heart sank, my mood went sour, I went into denial, then silently blamed it on QA, then I started fantasizing an error-proof system

Will there ever be one? Even if you planned meticulously, unit tested as much as you can; even if you had UI tests with selenium, JavaScript tests with Jasmine; even if you have QA tested every single feature, then run regression tests on the whole system; even if you tested your web application on all of the browsers, IE 9 &10, Chrome, Firefox ... errors still unexpected creep in.


If it is ever possible to eliminate bugs? And how much tests are enough?

Maybe bugs in code are as common and inevitable as flaws in human beings. 

Saturday, May 11, 2013

MVC has been having a great run for a while (a long while). It was introduced in the computer's stone age (the 70s, the first micro processor, the floppies, main frames), morphed into a well-accepted architectural theory in the bronze age (the 80s, IBM, computers for personal use became possible). Then computers and the internet and mobile devices became our overlords, exploding in power and complexity. To rein in the complexity and keep us from being entangled in the web of our own weaving, MVC becomes the default and de facto application architecture.

We all know now the three legs of the MVC stool:
Models - Data, business rules, logic, etc.
Views - Presentation layer, be it a chart, a table, an image gallery. The front end of things
Controllers - The link between model and view, the commander-in-chief that takes signals and issues command.
 


On the server side, we have a set of big players and well established design patterns and architectures. Ruby is the trail blazer that first goes MVC in full throttle. ASP .net MVC soon followed suit, moved over from the web form development model where UI is tightly coupled with the code-behind logic. ASP .net MVC provides powerful view engine, various routing techniques, and flexible controllers that accept and return data in the formats requested.

On the client/JavaScript side, the playground is always more diverse therefore more interesting and more confusing.

JavaScript MVC frameworks

Briefly searching the web on JavaScript MVC frameworks, you get a dozen and still counting. According to infoQ's research, the following are the top MVC frameworks:

Backbone.js: Provides models with key-value binding and custom events, collections, and connects it all to your existing API over a RESTful JSON interface.
AngularJS: A toolset based on extending the HTML vocabulary for your application.
Ember.js: Provides template written in the Handlebars templating language, views, controllers, models and a router.
Knockout: Aims to simplify JavaScript UIs by applying the Model-View-View Model (MVVM) pattern.
Agility.js: Aims to let developers write maintainable and reusable browser code without the verbose or infrastructural overhead found in other MVC libraries.
CanJS: Focuses on striking a balance between size, ease of use, safety, speed and flexibility.
Spine: A lightweight framework that strives to have the most friendly documentation for any JavaScript framework available.
Maria: Based on the original MVC flavor as it was used in Smalltalk - aka "the Gang of Four MVC".
ExtJS: Amongst other things offers plugin-free charting, and modern UI widgets.
Sammy.js: A small JavaScript framework developed to provide a basic structure for developing JavaScript applications.
Stapes.js: A tiny framework that aims to be easy to fit in an existing codebase, and because of its size it's suitable for mobile development.
Epitome: Epitome is a MVC* (MVP) framework for MooTools. soma.js: Tries help developers to write loosely-coupled applications to increase scalability and maintainability.
PlastronJS: MVC framework for Closure Library and Closure Compiler. rAppid.js: Lets you encapsulate complexity into components which can be easy used like HTML elements in your application.
Serenade.js: Tries to follow the ideas of classical MVC than competing frameworks.
Kendo UI: Combines jQuery-based widgets, an MVVM framework, themes, templates, and more.


Why so many frameworks? Why JavaScript MVC? What is MVC in JavaScript? Which one or ones to choose?

Well, if you were me, you probably would try to dismiss all those frameworks as noise. You may just want to bury you head in the sand, keep piling another jQuery plugin or another Ext js feature. But before long, the chorus of MVC became deafening, and you found your project on the JavaScript gradually spiraled out of control, and it becomes more and more difficult to track what triggers from what controls have changed the state of your data. The JavaScript become yet again become a multi-million lines of mess.

So think again about a JavaScript MVC framework.

Flavors of JavaScript MVC

MVC on the JavaScript side is a little different from the server side MVC you know, or the ASP .net MVC i know. In fact, MVCs are a little different from the JavaScript frameworks within. First and foremost, the frameworks try to clearly separate data and presentation, views and models.

Models: data that flows underneath your site. More precisely, data models that represents the structure/schema of your data. Data relationship, constraints can be defined and verified. Think of a user data model, it will have typical elements like age, gender, name, etc.

Views: layout, templates and interface. The side that your users can see and manipulate. They are your models marked up as photo galleries, task list, a table of data, etc.

Different frameworks have different takes when it comes to controllers. For some, the line between controllers and views are blurred, or views simply initiate the actions that theoretically would have belong to controllers. For example, backbone.js. Others do make controllers an command center that launches an application, initializes data models, monitors data changes and calls for appropriate actions. For example, Ext Js, Ember js.

There are also other flavors of JavaScript MVC. The most notably, the MVVM (Model-View-View Model) implemented by Knockout.js. Knockout.js has your typical views and models, however the controller part is termed as View Models, which bind DOM elements with data models and automatically refresh UI when data changes.

It is hard to get to know therefore evaluate each of the frameworks which all carry with them oh-so-good halos and incredibly cool demos and live websites, so we have to heavily rely on word of mouth that is circling on the internet. Some articles provide just that, for example:

Essential JavaScript: the top five MVC frameworks
Journey Through The JavaScript MVC Jungle

Backbone.js
Of the JavaScript frameworks, Backbone.js sits on top of the list and it has an impressive list of heavy-weight players, such as USA Today, LinkedIn mobile. Backbone.js has a collection of methods that manipulate data models, get/set, construct, make a collection, push/pop, extend. The whole backbone.js is revolving around data models and data handling logic.


var user = Backbone.Model.extend({
    name: 'Annoymous',
    age: 18,
    initialize: function() { ... },

    email: function() { ... }
});

Knockout.js
Knockout.js is particular hit with ASP .net MVC, I do not know why. And the heart of Knockout.js is view-model.

The following (from Creating a Dynamic UI with Knockout.js)populates a products array from data requested from a web API.
function ProductsViewModel() {
    var self = this;
    self.products = ko.observableArray();

    // New code
    var baseUri = '@ViewBag.ApiUrl';
    $.getJSON(baseUri, self.products);
}

Ext Js
Dealing mostly with Ext Js in my professional life, I have to give Ext Js MVC a mention. Ext Js 4 makes data packages center of its library, it also has an extensive set of data model mechanisms to define, validate, populate and manipulate data. As typical of Ext Js, it goes on the verbose mode. An Ext Js MVC application would have the following file structure:

 
 Ext Js Controller establishes awareness of its view through the init method:
Ext.define('AM.controller.Users', {
    extend: 'Ext.app.Controller',
 
    init: function() {
        this.control({
            'viewport > panel': {
                render: this.onPanelRendered
            }
        });
    },
 
    onPanelRendered: function() {
        console.log('The panel was rendered');
    }
});
And an Ext Js View use an Ext Js component for display.
Ext.define('AM.view.user.List' ,{
    extend: 'Ext.grid.Panel',
    alias : 'widget.userlist',
 
    title : 'All Users',
 
    initComponent: function() {
        this.store = {
            fields: ['name', 'email'],
            data  : [
                {name: 'Ed',    email: 'ed@sencha.com'},
                {name: 'Tommy', email: 'tommy@sencha.com'}
            ]
        };
 
        this.columns = [
            {header: 'Name',  dataIndex: 'name',  flex: 1},
            {header: 'Email', dataIndex: 'email', flex: 1}
        ];
 
        this.callParent(arguments);
    }
});
So which JavaScript MVC framework would you choose?

References: 

The MVC Application Architecture
Essential JavaScript: the top five MVC frameworks
Journey Through The JavaScript MVC Jungle
Backbone.js
Knockout.js

Monday, May 6, 2013

Ext Js has many features that rock. One of them is buffered grid. So unlikely traditional grid where pagination is done through navigating with a set of controls. Ext buffered grid panel allows you to scroll through the grid infinitely, the same way you would scroll up and down with a long web page. Behind scene, this is made possible by a buffered data store that fetches extra pages of data and cache them in page cache. A buffered store is configured as:
 buffered= true

However with this powerful feature comes with unanticipated bugs (oh, boy, do not we developers live with bugs.)

 The bug today I found is that buffered grid does not go well with grouping and column reordering. So using the sample infinite-grid provided by Sencha, I made a minor modification to have the grid group by author, and I get the following grid:

 

All is well, until I started to drag and drop to reorder the columns. Oops, after one successful drag-n-drop, the grid frozen.

After a little debugging, I found that the columnMove event triggered a refresh grid call, which in turn triggered a getGroups call, which in turn triggered an unhandled undefined error. StackTrace and error screenshot as the following:



So after some noodling around, I made the following changes and made the grid happy again.
 Ext.define('Ext.csx.data.BufferedJsonStore', {
    extend: 'Ext.data.Store',
   
    getGroups: function (requestGroupString) {
        if (!this.data.items) {
            return [];
        }
        this.callParent(arguments);
    }
});

Complete code available at: https://github.com/xunxdd/extjs.git

Saturday, April 27, 2013

In the book High Performance Web Sites, Steve Souders laid out 14 rules for better web performance. 

The rules are: 

1. Make fewer HTTP requests
2. Use a content delivery network
3. Add an Expires Header
4. Gzip components
5. Put stylesheets at the top
6. Put scripts at the bottom
7. Avoid CSS expressions (an IE-only evil)
8. Make JavaScript and CSS external
9. Reduce DNS lookups
10. Minify JavaScript
11. Avoid redirects
12. Remove duplicate scripts
13. Configure ETags
14. Make Ajax Cachable

That was 2007. 

Today, most of you probably would react to the rules with: So? I already know that. And most websites have adopted most of the rules. An iron proof how succinctly and effectively the rules are stipulated. 

On the heel of the success of High Perfomance Websites, in 2008 Steve Souders teamed up with other JavaScript/Front-end web development stars and published another book: Even Faster Web Site.

Only today I happened upon the book and hastily read it over. It has many gems here and there. Doug Crawford written about Axes of Errors and the AJAX landscape; Nicolas Zakas dives in various JavaScript performance tips. And many more.

There is one bit I am very interested in: the key of writing efficient CSS selectors, because normally to me, CSS is the most lightly treated element in a big application, it is mostly perfunctorily written and I hardly ever given it any thought on its performance implications.

First of all, a little open secret.
Browsers match CSS selectors from right to left.




Take for example: p a.active

So instead of shooting for your target from left to right, that is, finding all p then all anchor elements inside p, it sets out finding a.active first, then continue to their parents.

Why it goes about this way? StackOverflow has a very detailed explanation.

With this insight, some rules can be laid out for writing efficient CSS selectors. And yes, indeed, the rules have long been laid out by a guy named David Hyatt in a article Writing Efficient CSS for use in Mozilla UI.

The rules are:

1. Avoid universal rules 
 * -- wild card selectors are dangerous

2. Don't qualify ID selectors
   DIV #unique  is meaningless

3. Don't qualify class selectors
   Try change selectors such as LI .chapter to .listChapter

4. Makes rules as specific as possible
    Try not write selectors with 18 nested levels.

5. Avoid descendant selectors
   Descendant selectors are expensive.

6. Avoid tag-child selectors
Think twice before you use. Better just drop it.

7. Rely on inheritance.