TEERA 2.0

I’ve been a strong advocate for TDD for a bit over a year now. It’s the thing that I would suggest to anyone who cares about improving their software development and ask me about it. Now, the benefits of TDD are well-known, yet I find many people are still skeptic about taking the TDD pill. Here is my take based on my own experience with the technique, no fluff.

Testing discipline

We, developers, enjoy writing code but not so much of testing them. Now we all know that testing is good and we shouldn’t leave it all to the QA, but it’s another story to keep the discipline to do it. I find that if a developer leaves testing to later in development, he’ll end up either forget or not having enough time to do it. Since I started doing TDD, the minimum test coverage of my code’s gone up quite a lot (about 70% on a good day).

Enforce better design

With TDD, I have to think about how to test the code before thinking about the code itself. And to be able to unit test my code, I need to have a modular design. I also need interface based design. So I have to follow a good coding practice in order to effectively design my test. Many TDD followers would say that you’re already done designing your code when you are working on your test. I can’t agree more.

Peace of mind

“Any line of code is guilty until proven otherwise“. How confident are you on your code? Are you sure that your code does what it’s supposed to do? I find myself more confident in my code and it really helps me sleep much better at night.

“Whack-a-mole” proof

Whack-a-mole scenario describes the butterfly effect in software breakage. A break or a change in any line of code can trickles and bring down the entire system. When this happens, developers usually go after and do a clean-up op on the breakage as it is. But it can get worse when the act of fixing the breakage changes the behaviors of the code and introduces more breakages elsewhere in the system.

To prevent this disaster, you need a complete code testing coverage. This testing coverage will serve as guideline. When someone changes a line of code or check in new code, you can run this entire test suite again. If all the test cases pass, you can be sure that it’s still functioning as it should. More on this in Continuous Integration

Keeping up the morale

Developing test system, one test at a time give quick win goals and direction to work on. TDD serves as a self checking system that I’m on the right path of developing working component.  There’s also a sense of getting something done by the end of the day as well.

These points are based on my own experience. So, believe me, take the TDD pill it’s good for you.

I find Linq to comes in handy from time-to-time, especially when I refactor my code and try to shorten it. Consider this example, let’s say I have a Book data that I want to dump into XML document. Now I have a Book class to represent the object.

    class Book
    {
        public int ID { get; set; }
        public string Title { get; set; }
        public string Author { get; set; }
        public double Price { get; set; }
        public Book(int id, string title, string author, double price)
        {
            ID = id;
            Title = title;
            Author = author;
            Price = price;
        }
    }

To dump a collection of Book to XML, the old fashion way of doing it is creating foreach loop over Book collection and create XML object. With Linq it can become a one-liner.

            List<book> bookList = new List<book>();
 
            //Create book collection
            bookList.Add(new Book
                            (787, @"The Back of the Napkin: "+
                             @"Solving Problems and Selling "+
                             @"Ideas with Pictures", "Dan Roam", 16.47));
            bookList.Add(new Book
                (89, @"Presentation Zen: Simple Ideas "+
                 @"on Presentation Design and Delivery"
                 , "Garr Reynolds", 19.79));
            bookList.Add(new Book
                (897, @"C# in Depth: What you need to master C# 2 and 3",
                "Jon Skeet", 26.29));
 
            //Generate XML tree from Book collection data
            XElement xmlElement =
                        new XElement("Books",
                                bookList.Select(
                                    book => new XElement("Book",
                                                new XAttribute("ID", book.ID.ToString()),
                                                new XElement("Title", book.Title),
                                                new XElement("Author", book.Author),
                                                new XElement("Price", book.Price)
                                                        )
                                               )
                                    );
            Console.WriteLine("{0}", xmlElement.ToString());</book></book>

This query yields the XML output. No loop or messy enumeration required, the code is also more readable the Linq query.

From the feedback I got from ruby on rails course, there seem to be many students who find many core concept of ruby difficult to understand. So, the course mailing thread aside, I’ll try my best to explain some of these concept in depth here. This post will be about the concept of Mixin and ruby Module.

Mixin and Module

Wikipedia offers one of the best description of what mixin is.

“In OO languages, a mixin is a  class that provides a certain functionality to be inherited by a subclass, but is not meant to stand alone. Inheriting from a mixin is not a form of specialization but is rather a means to collect functionality. A class may inherit most or all of its functionality by inheriting from one or more mixins through multiple inheritance.” - Wikipedia

Ruby’s flavor of mixin comes in the form of Module. Module is a collection of methods and constant. The most commonly use of module is to have its members (methods, features) “mixed” into a class. The most evident example is the Kernel methods that are mixed-in to Object class. Because of this, methods from Kernels are universally available to all ruby objects.

Basic example

In the last post, I showed an example of ‘puts’ method. ‘puts’ is available to all Object in Ruby. ‘puts’ exists in Kernel module, and Kernel is mixed into Object class. Now since everything in ruby is an Object, they also automatically have access to ‘puts’ in Kernel module.

puts "string".class.ancestors
# [String,Enumerable,Comparable,Object,Kernel]

puts Object.class.ancestors
# [Class,Module,Object,Kernel]

puts Object.private_methods.include?('puts')
# true => Object has 'puts' method as member


puts Object.class.private_methods(false).include?('puts')
# false => 'puts' is inherited member


puts Kernel.class.private_methods.include?('puts')
# true => 'puts' is a member of Kernel module

Another example is the use of Math module. you can access Math’s constant value without having to include Math module

puts Math::PI
# 3.141592653589793

puts Math.sqrt(256)
# 16

Matz has called the mix-in feature of module “single inheritance with implementation sharing“. It’s a way of getting benefit of multiple inheritance without the mess and difficulties.

Module instance method

With “include” keyword, the modules methods become available  as object’s instance methods. You can think of including Ruby module to a class is similar to appending functions of that module to an object.

module MyModule
   def instance_method
      puts 'this is MyModule instance_method'
   end
end

class MyClass
   include MyModule
end

my_class = MyClass.new
my_class.instance_method #this is MyModule instance_method

As you can see, with ‘include‘ keyword, MyModule’s method becomes available to all instance of MyClass object.

Module class method with append_features

It’s also possible to make module methods available to class. What we’re going to use is ‘append_features‘. By overriding module’s ‘append_features' method with class as parameter, nested method under append_features will be accessible to that class.

module MyModule
   def MyModule.append_features(childClass)
      def childClass.classMethod
         puts 'MyModule classMethod'
      end
      super #HAVE to call super here
   end
 
   def instance_method
      puts 'this is MyModule instance_method'
   end
end

my_class = MyClass.new
my_class.classMethod     # illegal
MyClass.classMethod      # print MyModule classMethod

A few things to note here. First,’append_features‘ actually do the work equivalent to include operation. Therefore, we need to call super. Otherwise, the rest of the code in the module wouldn’t be included at all. Another thing, the nested method can only be singleton/static method.

Module class method with inheritance

Alternative to using append_features, you can mix in module’s instance methods as class methods with inheritance

class MyOtherClass
   class << self
      include MyModule
   end
end

MyOtherClass.instance_method
# print this is MyModule instance_method
# instance_method becomes class method

As you can see, the ruby language itself is very powerful. But by nature, it still follows the Object Oriented principle. On the future post, I’ll review the concept of blocks and lambda expression.

Less than a week since it began, many interesting discussions have been floating around ruby on rails course mailing thread. A great one in particular is the discussion on the OO-ness of ruby taken from the conversation between Victoria Pocladova and Raul Parolari. Consider this:

“Ruby is an object oriented language and an object has a well-defined interface that specifies the behavior of the object in a manner that is independent of its implementation. This interface defines the collection of services that can be invoked by other objects.” - http://c2.com/cgi/wiki?ObjectOriented

Main.rb and Object

The next questions become how would you describe the behavior “main.rb”? You can have blocks of code within the main class, and no method containing this code. How is this code exactly invoked by another objects, or by the compiler itself when the main class is executed? And since I don’t have any methods in the main class, where does the “puts” command come from? Does main class get special treatment like void main() method in Java?

It does seem a bit contradiction to the earlier paragraph describing the OO concept of ruby. Raul Parolari, my course adviser colleague, offers a great explanation of this.

First, let’s try creating a main.rb file and execute this line of code.

puts self #main
puts self.class #Object

‘main’ is like the backstop of Ruby program. It is the farthest back you can fall; but it is also the current object as soon as your programs starts up. By the way, ‘puts’ is in Object? no, it is a method of module Kernel. But wait a moment; didn’t we say that ’self’ class was Object? how can we call ‘puts’ on an Object instance? Because… every object’s search path includes the Kernel module; and how is that?

Kernel module

That’s because the class Object mixes in Kernel, and every object’s Class has Object as an ancestor. Now try this

puts self.class.ancestors #[Object, Kernel]
puts 'string'.class.ancestors # [String, Enumerable, Comparable, Object, Kernel]

Again, we see Kernel lurking at the back.

puts

The next thing we can investigate is if ‘puts’ is actually a member of Object.

puts self.class.private_methods.include?('puts') #true
puts Object.private_methods.include?('puts') #true

Oh wait? Didn’t I just say that ‘puts’ is a method of module Kernel? If you trace EVERY class in ruby, you’ll see Object and Kernel at the top of inheritance hierarchy. Now let’s try to interogate Object class again and keep out inherited members.

puts Object.private_methods(false).include?('puts') # false
puts Kernel.class.private_methods.include?('puts') #true

Gotcha! As you can see, ‘puts’ is actually a member of Kernel module that Object benefits from. And since every class in ruby is an Object, they all benefit from the work of Kernel as well.

As the lead adviser for this program, I’m astound to see this kind of thing in the discussion thread myself. The understanding of core concept highlights beauty and benefits of the language.

I ran into a situation that I have to write JavaScript that execute only when the page is posted back. Currently there’s no straight forward way in ASP.NET Ajax to check IsPostback from JavaScript. If you google around, you might find various way of getting this to work, like this one.

After some thought, the easiest hack I came up with is this.

function IsPostBack() {
   return <%= IsPostBack.ToString().ToLower() %>;
}

I’ve recently been appointed as the lead adviser for JavaPassion.Com’s Ruby on Rails online training course by the legendary Sang Shin. I talked about this Ruby on Rails online course in my previous post. At the time of this writing, there are 998 registered students and will likely to hit 1K in a few hours. As the lead adviser, I’ll basically be the go-to guy for students when it comes to questions and assist Sang in developing course content.

I must say that I am no expert in RoR. I picked it up only about a year ago as a hobby. Since then my experience and relationship with Ruby and Rails communities have been nice and warm. The same goes for Java and other open source communities. Quite often that I think about giving back and contribute to open source projects. However, neither my coding skills nor my schedule had allowed me to do so. So my resolution becomes

“If I can’t make a contribution in code, then I’ll do it in the form of content!”.

And here it goes…

Reminder: the course starts July 15th, go register if you havn’t done so!

In part 1 of this post, you can see how you can encrypted your ViewState and ensure its integrity across postbacks. A few more things you can do to beef up your ViewState security.

ViewState validation with ViewStateUserKey

It is possible for someone who got a hand on your ViewState data and reuse it at some other time or different user session. This scenario is what we call ‘One-Click attacks‘, a variation of XSS. ASP.NET provides ViewStateUserKey as a way to ensure that ViewState data is tied to specific user session.

In short, ASP.NET can use ViewStateUserKey for encryption salt. So if we set ViewStateUserKey to current session ID or specific user information, this ViewState data will not be usable to other users or sessions. For more detail explanation, check out Scott Hanselman’s posts here and here.

Note that setting ViewStateUserKey should be called early in the OnInit method, before ViewState is encrypted.

Taking ViewState out of Page output

The ultimate measure to secure ViewState is to remove it from rendered page altogether and store it somewhere else. Apart from tighten up security, another benefit is reducing the size of output HTML page.

ASP.NET uses a page state persister class to take care of persisting ViewState data. By default, ASP.NET page uses HiddenFieldPageStatePersister, which as the name implies, persist ViewState to page’s hidden field. Another default persister is SessionPageStatePersister which keeps page ViewState in current session.

To switch to use session persister, add <sessionPageState> in Web.config file. You can also specify the size of maximum saved ViewState.

Custom page ViewState Persister

ASP.NET 2.0 allows you to create custom persister by overriding PageStatePersister class. In the code sample, I create a persister that save ViewState in a separate text file.

Subclassing System.Web.UI.PageStatePersister

Next, we have to override Save method. This method is called when ViewState data is persisted to storage medium. In this case, I persist ViewState data to Text file.

Generate GUID for ViewState persisting file name

Note that I keep GUID in a hidden field. We’ll need this GUID when we load back ViewState on postback.

Writing ViewState data to text file.

You’ll see on the output HTML that __VIEWSTATE hidden field is now empty! We only keep GUID in a hidden field to refer back when we load ViewState data. This helps reduce page size significantly if your page relies on heavy ViewState data.

Lastly, override Load method to get ViewState value back from persistence medium.

Now you need to tell your aspx page to use this custom persister. To do that, simply override PageStatePersister property on the page.

You can download the code for this class HERE.

Where should I put ViewState data

Putting ViewState in Session has some disadvantages. First, you can lose session ViewState if it reaches historySize limit (if you open multiple windows of the same web page). You may also run into a memory problem if ViewState is very large. Consider when you have a lot of users with large ViewState data, all these data is kept in web server memory. From the sample code, persisting ViewState to external text file is effective against large ViewState, yet it can incur significant file IO cost. With ability to create custom page ViewState persister, you can go as far as persist it to SQL database

I didn’t pay much attention on the security aspect of ASP.NET ViewState until a few days ago when I got to use my friend’s labtop. After browsing around, I could see that his web browsers cached some pages from ASP.NET website. So just for fun, I quickly downloaded ViewStateDecoder and decoded the cached page’s leftover ViewState. Here’s what I found:

The blackened part is my friend’s email

Now this cached page was from an online shopping site in Asia. With information like username and email, a moderately evil geek can do quite a lot of financial damage to my poor friend.

As ASP.NET developer, ViewState is one thing that we grow to love and hate. Love - it’s the key technique that make stateful ASP.NET controls work on stateless HTTP protocol! Hate - it’s heavyweight and expose a security hole. Here’re a few things you can do to protect ViewState from malicious attacker.

Enable ViewState MAC

MAC - Machine Authentication Check, is a mechanism to ensure the integrity of ViewState data (data received is the same as data transmitted). Before ViewState data is sent out, ASP.NET quickly computes hash data from the ViewState and append that hash to the end of ViewState. When Postback occurs, ASP.NET deserialized returning ViewState and compare the original hash value to the returned hash value. If both values are NOT the same, the ViewState data has been corrupted/changed.

Enabling ViewState MAC is very easy. You can either do it for a specific page.

Or you can apply to every pages through Web.config file

By default, MAC hashing algorithm is SHA1. You can change the algorithm by setting validation attribute of machineKey section in Web.config file.

You can also encrypt it by selecting Triple DES (3DES) algorithm

ViewState Encryption

If you select “3DES” or “AES” as machineKey validation attribute, your ViewState will get encryption automatically. Specifying encryption algorithm in Web.Config file will apply to all pages. You can enable/disable encryption on page basis as well by setting ViewStateEncryptionMode. By default, the value is Auto.

An extra hidden field will be added for a page with encrypted ViewState. This marker field tells ASP.NET that the ViewState hidden field has to be decrypted on postbacks.

Alternatively, you can programatically request ViewState encryption from C# code. The case that you’ll need this is when you have ViewStateEncryptionMode directive set to ‘Auto’ and want to request encryption for this page from code. ASP.NET 2.0 introduced a new Page.RegisterRequiresViewStateEncryption() method. Note that you should call this method before LoadViewState event for postback (call it in Init method).

By default, ASP.NET generate random key for encryption. If you want a stronger encryption or you are working in a web farm, you’ll need to specify your own validation key. The validation key is a string of 20 to 64 random, cryptographically-strong bytes, represented as 40 to 128 hexadecimal characters. The longer the key the better.

The easiest way to generate this key is to use System.Security.Cryptography.RNGCryptoServiceProvider class.

The keyLength parameter can be 40 or 128. This method returns cryptographically strong validation key.

Once you get the validation key, set it for machineKey in web.config file. If your application is hosted in a web farm, make sure that the validation key is consistent across all the servers.

In the part 2 of this post, I’ll show you how to user ViewStateUserKey and create custom ViewState persister that allows you to remove ViewState data from rendered page entirely.

We often use schema to validate XML documents. However, there are times when there’s no schema available for existing XML data. I found out just recently that .NET framework can actually do this for me! So this is another “Oh..I didn’t know I can do that” post.

The key here is to use System.Xml.Schema.XmlSchemaInference class.

Using XMLSchemaInference to retrieve XSD schema from existing XML content

As you can see from this code, you can get schema by simply call InterSchema method with XMLReader as parameter. The XMLReader parameter should contain the XML document/fragments that you want to generate schema for.

I created an XML utility from the code above. Try running the application with this example data.

The output schema looks like this

<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Items">
<xs:complexType>
<xs:sequence>
<xs:element maxOccurs="unbounded" name="Item">
<xs:complexType>
<xs:sequence>
<xs:element name="Name" type="xs:string" />
<xs:element name="Price" type="xs:decimal" />
</xs:sequence>
<xs:attribute name="id" type="xs:unsignedShort" use="required" />
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="year" type="xs:unsignedShort" use="required" />
</xs:complexType>
</xs:element>
</xs:schema>

The more specific XML data given, the more detailed schema is generated. Letting the framework generate schema should not be the end solution, it merely gives you a rough draft for the schema that you can then add and refine later.

A friend sent a URL for Nerd Test to me. She got this from someone else and thought of me (..nice). Well, I know for a fact that I’m pretty geeky but never thought of myself as a Nerd. Sadly..here’s my score

High-Level Nerd. You are definitely MIT material, apply now!!!.

Go check out your nerd level here: http://www.nerdtests.com/ft_nq.php