Interested in a blog post on this? Take it to the comments section.

Cheers,
@mkennedy

Want to get ready for this class or a class like it or just get a quick-start on .NET? Here’s what I would recommend. There are many books covering the topics (but no small list covering most). Instead of trying to read them all in-depth (a fairly unattainable goal), I would suggest that you read the first 2 – 3 chapters from all of these books listed below. This would cost about $250 if they buy them all, but they are all available on Safari Books Online.

http://my.safaribooksonline.com/

You can get the free trial for 10 days and if you like it you can then pay $19 / month and have access to all these books (see the coupon at the bottom of the trial page on safari online).

CLR via C#, Third Edition (foundational knowledge)
http://my.safaribooksonline.com/book/programming/csharp/9780735640467

WPF 4 Unleashed
http://my.safaribooksonline.com/book/programming/microsoft-wpf/9780132174367

Silverlight 4 in Action
http://my.safaribooksonline.com/book/web-development/silverlight/9781935182375

Pro ASP.NET MVC 3 Framework, Third Edition:
http://my.safaribooksonline.com/9781430234043

Programming Entity Framework
http://my.safaribooksonline.com/book/software-engineering-and-development/9780596807276

RESTful .NET
http://my.safaribooksonline.com/book/programming/microsoft-dotnet/9780596155025

Learning WCF
http://my.safaribooksonline.com/book/programming/microsoft-wcf/9780596101626

C# in Depth
http://my.safaribooksonline.com/book/-/9781935182474

jQuery in Action
http://my.safaribooksonline.com/book/web-development/jquery/9781935182320

Host Your Web Site in the Cloud
http://my.safaribooksonline.com/book/web-development/9780980576832

MongoDB: The Definitive Guide
http://my.safaribooksonline.com/book/databases/mongodb/9781449381578

Agile Principles, Patterns, and Practices in C#
http://my.safaribooksonline.com/book/programming/csharp/0131857258

Debugging Applications for Microsoft .NET (sections on WinDBG)
http://my.safaribooksonline.com/book/programming/microsoft-dotnet/9780735645349

Cheers,
@mkennedy

Slides: Kennedy-Rich-Forms-with-ASP.NET-MVC-Slides.zip
Demo Application: Kennedy-MVC-Forms-Demo.zip

Video Recording:

Remember, you’ll need to run MongoDB to make the web app work. Read the readme in the zip file for details (it’s easy!).

Cheers,
@mkennedy

Want to learn more about building great forms in MVC? Attend my free DevelopMentor webcast this Tuesday.

Webcast: Building Rich Input Forms in ASP.NET MVC

https://www3.gotomeeting.com/register/559059054

In this webcast we will explore the powerful features of ASP.NET MVC that allow us to build rich forms that accept user input. We’ll begin by discussing the built-in HTML Helpers and Model Binding. Next we’ll add validation and show how we can do both client- and server-side validation using DataAnnotations. Finally, we’ll see that sometimes using domain models as our form-bound objects doesn’t make sense. We’ll look at more advanced scenarios using View Models.

Cheers,
@mkennedy

Let me give you one less reason choose the rocky-road of desktop apps today. You’ll hear people tell you that if you build applications for business users (so-called LOB apps) that are input heavy or have complex navigation, your only choice is to build a Windows application.

Why? Because your power users will want hot-keys. They don’t want to use the mouse and navigation and all that stuff that makes the web less usable. They’ll just want to hit a few keystrokes and jump from place to place and do that quick look-up or data entry.

If you haven’t looked carefully around the web lately, here’s a revelation:

Real web apps have hot-keys too.

Just look at Twitter, GMail, GitHub, Hotmail, and many more. They all have rich hot-key support (although with GMail you’ll have to turn it on). Scott Hanselman has a great article on this: Are you using the Web’s Keyboard Shortcuts and Hotkeys?

So if Scott’s is so good, why write this one? His post is meant to open your eyes to how web apps can be. I want to show you, Mr. / Ms. Software Person, how easy it is to get there. You might be thinking to yourself, “Sure Google & Twitter can do this because they have millions of dollars and a bunch of developers. My project has 2 people and a 1 month deadline.”

It turns out, adding hot-keys to a web app is crazy easy. Especially with the jQuery plugin jQuery.hotKeyMap.js which I recently wrote. Here’s the blog post I did on it. You should check out the sample application I built to see it in action.

So give hot-keys in your web apps a try. Feel free to use my jQuery plugin and sample application as a starting point. Once you get over the hurtle of teaching your users to use them, you’ll hear a whole lot less clamoring for desktop apps. All the pain from weird errors on certain versions of Windows, installers, patches, admin rights, .NET Framework versions and other unpleasantries that you’ve gotten used to, because you had to, will be distant memories.

Cheers,
@mkennedy

Of course, you can do this by adding target=”_blank” to all your external links, but that’s tedious to say the least. So here is a simple jQuery plugin I created to do the trick. (Yes I know there are others, but none seem quite right, so here is yet another.)

Download: Query.externalLinks

The usage is pretty easy. Pick a section to scan for external links and say go:

Here is a simple example (from my previous post on hotkeys). Click on the title or the twitter icon in the lower right to see it in action.

Currently, it works by adding target=”_blank” to all fully-qualified links. For example, the first link will be a new window, the second won’t:

<a href=”http://www.google.com”>google</a>
<a href=”/contact”>contact me</a>

Hope you find it helpful.

Cheers,
@mkennedy

Expect a more general blog post on the topic soon. For now, I’d like to share my jQuery plugin that enables this capability with you.

Announcing jQuery.hotKeyMap.js: A jQuery plugin that allows you to add complex hot-key functionality and navigation to your web apps with very little effort.

You can download it here:

jQuery.hotKeyMap

It’s built on top of John Resig’s jQuery.hotKeys. His plugin is great (as you’d expect), but as far as I know doesn’t support complex series of keys. For example, if you want to GMail style navigation (press ‘g’ then press ‘t’ to go to sent mail), that’s fairly difficult using his plugin. Here’s how you accomplish this with my plugin:

In the code above, we’re assigning three immediate key actions (e.g. ctrl+z executes undo). We are also adding one complex sequence: g, h => navigates to the home page (calls navTo, not shown, passing ‘h’).

Want to see it in action? Check out my sample application:

Web Apps Have Hot Keys Too!
http://www.michaelckennedy.net/Samples/blog/HotKeys/

It even comes with Visual Studio intellisense:

I hope you find it useful!

Cheers,
@mkennedy

   Kennedy-Demo-Amazoon-Sound-Sound-Usergroup.zip

The topic will be:

BUILDING WEB APPS WITH ASP.NET MVC AND NOSQL

This talk guides you through building modern web applications using ASP.NET MVC and MongoDB, one of the most popular NoSQL databases.

You will learn some best practices for getting started with MVC. We’ll cover building rich-forms to accept user input. And if time permits, we might even add some client-side techniques using jQuery and MVC.

All of this will be built upon the powerful non-relational database MongoDB. We will discuss the origins of the so-called NoSQL movement and why you might choose a non-relational database over SQL Server. You’ll also see our data access layer will be built using LINQ to MongoDB.

Of course, you won’t be in for a night of PowerPoint. This talk is a series of interactive demos using Visual Studio 11, Windows 8, and C#.

I hope to see you there! There is a chance I’ll have some prizes from DevelopMentor to hand out while I’m there too.

Cheers,
@mkennedy

Apple touts 4G as finally giving you the ability to watch HD video streaming on the go. They’re implying that finally you are free of 3G slowness and now you have high-speed Internet. You can do normal high-speed Internet things on the go. You can even tether up to 5 devices to that connection.

But I wouldn’t do too much of that if I were you! You see, this capability (while awesome) reminds me of that Prince Bride quote:

You keep using that word.
I do not think it means what you think it means.

Yes, you can watch HD video. But only 3 hours of it, per month. And that’s it. At the highest premium plan ($50/month). The starter plan gives you maybe 30 minutes / month. That had better be pretty good video, because it’s not cheap.

Here’s the relevant excerpt from the apple order page:

Yikes! Don’t despair though. There are other options. I’m currently using Clear.net. For $40/month they give you UNLIMITED data plans.

Here’s my recent usage graph. I usually do not use my clear at home, but do use it when I’m working out at Starbucks or elsewhere. Keep in mind, this graph is PER DAY USAGE and only counts 8 days of this period. I’ve also highlighted where the MONTHLY iPad 4G plans would land:

You can see several days of normal usage would bust the monthly limit of the premium plans. Most days that I have used the device all day would bust the mid-grade plan (not the cheapest, which is 250 MB, but the 3 GB plan).

Yes, there is the draw-back of carrying an extra device (even though it’s very small – credit card sized). But you get 4G and you get to use the Internet as if you really had high-speed Internet. Plus, you get a wireless network supporting up to 8 devices at once (maybe 9 with a hack).

Here’s a speed test from a random day on clear 4G. You can see it’s pretty speedy, but I have seen it above 10 Mbps.

So like many of you, I’m eagerly awaiting March 16th when the new iPad ships. But I’m just as happy it comes with WiFi and not 4G. I already have the covered.

Cheers!
@mkennedy

1. First of all, even MVC 3 has old-and-crusty aspects lurking in its projects. There are old MicrosoftMvc*.js AJAX and validation libraries that have be replaced with new jQuery hotness. These *.js files aren’t used so just delete them.

2. Many of the dependencies of your MVC project are out-of-date as soon as you create your project. You have an old version of jQuery, Entity Framework, etc. Luckily Phil Haack and crew had the brilliant insight to link these to NuGet. So the next thing you do is just run NuGet and choose the Updates tab.

What’s that, you don’t have NuGet? Oh, you need to fix that right now!

Here’s what you can expect under the NuGet updates page:

3. Speaking of NuGet, by default all the external, NuGet-based JavaScript files get mixed in with your few JS files you always need to go back to for editing. So this is what you do:

Create a separate folder called js and put your JavaScript files in there.

Don’t forget to import site.js into your _Layout.cshtml file.

4. You like intellisense, yes? You also like fast-loading pages yes? So on one hand, you want to use jquery-1.7.1-vsdoc.js for intellisense, but you want jquery-1.7.1.min.js for performance. Here’s what you do. Import the minified version into your layout page (see later in the article) but add a “reference” to the vsdoc version in your site.js file. See the /// <reference> at the top? See intellisense? Good!

BTW, you are using namespaces or the prototype pattern or something besides just importing junk into the global namespace aren’t you? I thought so.

5. Next, unless you are planning on using jQueryUI you should remove it via NuGet. It has so many files under the content folder that unless you plan on using it. They are just mental weight. Everything in your project should earn its keep. You can always add jQuery UI back via NuGet if you want it.

6. Web programming is awesome, except when it’s not. That usual happens when you find your page looks different in IE vs Chrome vs Safari. You can avoid some of these issues just by referencing Eric Meyer’s reset.css. Be sure to put reset.css as the *very* first CSS file.

7. In ASP.NET MVC, there are models, then there are models. In MVC, there are domain models that are usually UI independent. Then there are models specifically meant to be tied to views. These view-oriented models are called View Models. I recommend you put view models in their own folder to make this dead-obvious.

Stephen Walther has more on View Models if this is the first you’re hearing of them.

8. You said you like fast pages before right? Well MVC is “doing it wrong” in order to allow you to inline JS which you should not do. Put the JS imports at the bottom of the page.

Be careful though, modernizr.js needs to stay at the top to work.

9. There are times you need to conditionally import JavaScript files from pages. The technique above in #8 makes this adhoc importing difficult. We can easily fix this by adding a layout section called AdditionalJsImports and make it optional (see above). This lets pages add more JS *after* jQuery.js and site.js.

Conclusion

If you do these 9 things at the start of your project, you’ll be on a good path to a well-architected system with some key advantages: It doesn’t have files in the wrong places, it doesn’t have unused or outdated files, and it is built for performance.

Hope you find these tips useful! If you have your own, please add them to the comments below.

Cheers!
@mkennedy

Usually, these view files are organized into sections by controller which keeps this manageable. For example, your solution might look like:

• Controllers
• Models
• Views
  • Home
    • Index.cshtml
    • About.cshtml
  • Products
    • All.cshtml
    • Show.cshtml
    • New.cshtml
  • Shared
    • …
  • etc…

So that any one folder might only have 10 cshtml files. But there is one folder that is disproportionately crowed, that Shared folder where all the common partial views and layouts are kept. A quick look at one of my recent MVC projects shows 113 files in that shared folder!

A 113 files tossed into a single folder is painful to manage and difficult to find what you’re looking for at a glance. If they could be categorized into subfolders, say things like Shared/Navigation, Shared/Accounts, Shared/Users, things would be good again.

Presently MVC doesn’t look into subfolders of the shared views folder. But with a little open-source library I wrote and a few clicks on NuGet, it will!

Introducing Enhanced View Locations for MVC

To address this issue, I’ve created a project over at CodePlex:

Enhanced View Locations for MVC

http://mvcviews.codeplex.com/

With this library added to your project (via NuGet), your massive Shared folder can look like this:

Here’s how you use it (from the codeplex project documentation):

Enhanced View Locations for MVC allows you to further organize your ASP.NET MVC views without your action methods or Html.RenderPartial / Html.RenderAction elements knowing or caring about how they are organized or re-organized.

1. Begin by adding a reference to EnhancedViewLocations.dll to your ASP.NET MVC project via NuGet (http://nuget.org/List/Packages/EnhancedViewLocations) or by directly downloading and referencing it.

2. Next, define and register custom view locations in the form partial Uri’s from within the Global.asax file:

protected void Application_Start()
{
    AreaRegistration.RegisterAllAreas();
    RegisterGlobalFilters( GlobalFilters.Filters );
    RegisterRoutes( RouteTable.Routes );

    // Register the extra locations for views.
    RegisterCustomViewLocations();
}

private static void RegisterCustomViewLocations()
{
   // We can optionally enable the standard Razor view CS
   // (or VB) locations so we don't always search our
   // extra / custom areas first when searching for a view.
   // EnhancedViewLocator.EnableStandardRazorCSLocations();

   // Next, add two extra locations to store / arrange our views.

   // 1. in the templates folder under the standard shared view
   // folder.
   EnhancedViewLocator.AddFolder( "views/shared/templates" );

   // 2. in the navigation folder under the standard
   // shared view folder.
   EnhancedViewLocator.AddFolder( "views/shared/navigation" );

   // Finally, have the locator install a custom view
   // engine to manage the lookups.
   EnhancedViewLocator.Install( ControllerBuilder.Current );
}

I’ve recently signed up as an INETA speaker. If you would like me to speak at your user group or event, check out my INETA Speaker Profile or simply contact me directly.

I typically speak on NoSQL, MongoDB, ASP.NET MVC, Modern Web Applications (whatever that means!) and design patterns but I’m also open for other requests.

Cheers!
@mkennedy 

Enjoy!
@mkennedy

PS – Yeah, it’s from last year. I forgot to publish it on my blog so here it is…

[Update: Want to watch this as a screencast rather than article?]

In this ASP.NET MVC Foundations article, we’re going to look at building an ASP.NET MVC page which allows users to create and edit objects in our domain. We’ll cover just the basics of using HTML helpers to map model properties to our HTML form and Model Binding to convert our HTML form back into our rich domain object.

We’ll start with a very basic store website (downloads here: BasicMvcForms_starter.zip and BasicMvcForms_final.zip)which has a database and some basic products already listed:

Notice that we have five products. There links to edit and create products. Currently, they don’t do much:

First we’ll need to add action methods. That’s easy enough. Here’s the create code:

And similarly, the edit code:

Now that we have action methods for edit and create, we’ll need the corresponding views. Use the Visual Studio tooling to create two empty views as follows:

In the resulting dialog, choose strongly-typed with Product and an empty view without referencing the scripts (we do this globally already). In practice, you might choose “edit” and “create” to help jump-start the Razor code. In this example, we’ll do that from scratch so empty is what we want.

Now our links to edit and create products work. However they don’t have any content in their views. We’ll use the HTML Helper methods to convert our product into forms ready for the editing.

It all starts with Html.BeginForm(). We’ll define a form using this MVC convention and helper method as follows:

Next, we use the Model property of the view and the HTML helper methods to define the input fields. Note that we’re using Html.TextBoxFor() and Html.LabelFor() to create the fields. Our Featured property is a Boolean, so we’ll use Html.CheckBoxFor() on that one.

Once we flesh out the other properties, we’ll be finished with the create view. And it turns out the edit view is identical. There are tricks to share them across actions (e.g. PartialViews) but for our simple example, we’ll just copy / paste between the two views.

Last thing we need is a submit button to submit the form.

Now we should have a nice usable form to create products (or edit them if you copy & pasted that view). Here’s the edit view:

The final step is to capture the form data on the controller methods and update the database. This is where it gets interesting.

We will define a second method for each action (create & edit) which accept the post. We do not want our “show the form to start editing” code to mix with the “save the data and move on” code. We’ll achieve this separation using two attributes HttpGet & HttpPost.

Notice the original Create method has the GET attribute. This displays the form to begin creating a product. The second one is more interesting. It adds the product to the DB and returns to the product list. It also only accepts POST requests. Notice that it accepts a Product parameter which is populated using model binding by ASP.NET MVC.

Learn this pattern! It’s super common in MVC. I’ll call it the Get+Post+Redirect pattern.

1. HttpGet method shows form, returns View().
2. HttpPost method accepts the model (which is populated using model binding)
3. HttpPost method then updates the data and redirects to a new view.

Edit is similar. Often, websites use AutoMapper to do the manual copy / update you see here.

Now we have a fully functioning store (albeit a simple one). One glaring omission is validation. We’ll cover that in another post.

Conclusion

To wrap up, we took a basic MVC website and went through these steps:

•  Added edit and create methods
• The new methods returned the correct model
• Added strongly-typed views for each method
• Used Html.BeginForm() and related Html.XXXXFor() methods to build out the form
• Added a submit button
• Implemented the Get+Post+Redirect pattern in edit and create.
• Take the rest of the day off.

Good luck with your websites and happy POSTing.

- Cheers
@mkennedy

In this second installment, we’ll be covering exactly that. How do we manage having Visual Studio and associated tools and servers universally accessible in the cloud, even on mobile devices such as iPads?

We’ll start with the king of the cloud:

Amazon Elastic Compute Cloud (Amazon EC2)
http://aws.amazon.com/ec2/

At EC2 we can create a variety of Windows instances, get full admin access via remote desktop, and install anything we want (Visual Studio, SQL Server, MongoDB, etc). We can also run these for just 30 minutes here, 2 hours there and we’ll only be charge for the time we use it.

We are going to setup a Windows Server in EC2 which we can access from our Cloud OS, or any other machine or mobile device, and deck it out with all the developer tools we want. You have to choose your server type based on what you’re planning on doing as well as how much you want to pay. Here are the available options and pricing. You can roughly expect to pay $0.50/hr for a good workstation, or $0.12/hr for an acceptable but small one.

Amazon EC2 Instance Types: http://aws.amazon.com/ec2/instance-types/
Amazon EC2 Pricing: http://aws.amazon.com/ec2/pricing/

I’m currently using a Micro Instance for an SVN server which is always running as well as High-Memory Extra Large Instance for my development machine that I start and stop on-demand when I need to get some programming done. Here are the specs for the developer workstation:

High-Memory Extra Large Instance

17.1 GB of memory
6.5 EC2 Compute Units (2 virtual cores with 3.25 EC2 Compute Units each)
420 GB of instance storage
64-bit platform
I/O Performance: Moderate
API name: m2.xlarge

You have to admit, that’s a pretty killer workstation. It’s a overkill on the memory, but has better processors than the cheaper ones which also have decent memory.

The price of the dev machine is $0.69 / hour. I’ve also built some tools which help shut down the machine in case I get distracted. So far I’ve been using it pretty heavily this past few weeks and have spent probably $13 on it. That’s not bad for the possibility of being anywhere in the world (with an Internet connection) and getting my workstation running in an instant. OK, maybe it takes longer than an instant. To be fair, it usually it’s 10-15 seconds .

Take the Free Lunch

One thing you should do straight away if you’re going to try this is sign up for the EC2 Free Tier:

AWS Free Usage Tier
http://aws.amazon.com/free/

This won’t cover the EC2 CPU time (that’s Linux only), but it will cover storage, bandwidth, S3 snapshots, etc. You also get a free Linux machine to play with if you like.

Setting up Windows in the Cloud

OK, so if you’re chosen an instance type and are ready to start, you should do the EC2 walk-through I put together here to get your machine up and running and login for the first time.

Building Windows Machines in Amazon EC2
http://www.michaelckennedy.net/blog/2010/01/31/BuildingWindowsMachinesInAmazonEC2.aspx

I’d guess that takes about 15 minutes. When you’re done, come back here…

Installing Software in EC2 and Windows

So you’ve logged in and you want to install Visual Studio, SQL Server, and other MSDN goodies. The easiest way to get them is to use the ISO DVD images from the MSDN download site. In order to mount them on your EC2 machine (which has no DVD drive), you should install

Virtual CloneDrive
http://www.slysoft.com/en/virtual-clonedrive.html

which lets you mount ISO images as if they were proper DVD drives. Then you can download and install all the MSDN files you need. A good place to start is with Visual Studio:

Microsoft Visual Studio 2010 Premium Trial – ISO
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=f81412a2-d48e-4040-9b32-27eaf771c5db&displaylang=en

One word of warning here: I’ve noticed that Chrome can freak out and fail to download very large files inside your EC2 instance. Don’t know why, but it does. So be sure to use IE or Firefox to do the downloads for the ISOs.

Syncing your Files Between your Local Machine and Dev EC2 Machine

There are two techniques we can use to sync files, depending on what category of files we’re sharing and how. For the source files, you can always setup an SVN Server (that’s what my micro instance is about). I’m a fan of Visual SVN. It’s free and easy to setup and admin.

VISUALSVN SERVER
http://www.visualsvn.com/server/

Then on your EC2 dev machine (and your others you may have) setup:

TortoiseSVN
http://tortoisesvn.net/

and

AnkhSVN for Visual Studio Integration
http://ankhsvn.open.collab.net/

Then you have super easy, reliable, and secure sharing of your source files.

For everything else, you can just setup a Dropbox share between your Cloud OS and you EC2 Dev Machine:

If you don’t have Dropbox and want to try it, use this link to create an account

http://db.tt/Qo7m9PA

and you’ll get me some extra free space which I always appreciate. Then if you have a file on your local computer you want accessible on your cloud dev machine, just drop it in your Dropbox folder on the originating computer and it’ll be there as soon as it uploads.

Waste Not, Want Not

Now here is a problem you will encounter:

You’re working on your dev machine in EC2, something comes up and you get distracted and forget about it for the rest of the day. When you come back the next day you see your EC2 system is still running and you’ve wasted $12 by leaving it running idle overnight. That sucks. If it happens on Friday, it might run the whole weekend which is even worse.

To avoid this, I wrote a little tray utility that watches for extended idle times and will shutdown your EC2 instance if it’s idle for too long.

Below is how you configure it. Simple, right?

Add it to your startup menu to make sure it starts when you log in.

You can download it here. I may blog about this app in greater detail later, for now feel free to use it.

Shutdown on Idle
http://www.michaelckennedy.com/Downloads/Tools/ShutdownAfterIdle.zip

Reliable Access to EC2

One annoyance in EC2 is that every time you turn off your EC2 computer, it will take on a new random public DNS name. If you really want to access to your EC2 machine from anywhere, you probably want a reliable DNS name / IP address. That’s where Elastic IPs come into play in EC2:

Feature Guide: Amazon EC2 Elastic IP Addresses
http://aws.amazon.com/articles/1346

If you create an elastic IP, then you can map it to a sub-domain in your domain and always use that. For example I could register it for cloud-developer.michaelckennedy.net. Then as long as I have a way to start my machine and re-associate the IP, all my config on my various machines (such as remembered RDP settings and passwords and such) will “stick”.

Be aware that while your EC2 machine is off, you’ll be charged a small fee (few dollars / month) for the address. You can decide whether it’s worth the trouble to save the few dollars.

Mobile / iPad Access

I’m sure when you think of accessing your EC2 dev machines, you’re thinking of doing that form Windows or maybe a Mac (using  http://www.microsoft.com/mac/remote-desktop-client). But the iPad (and even the iPhone) has apps to allow you to use Remote Desktop to EC2. For example, here’s editing the code for http://chatpast.com in Visual Studio via my iPad:

(click for full size image)

And you can even bring up the keyboard or attach a Bluetooth keyboard.

(click for full size image)

The app I was using is called “Jump”, and you can find it here:

Jump for iPad
http://jumpdesktop.com/

Summary

I hope that gives you some inspiration to try a new way to work in the cloud. You saw that we can create fairly serious EC2 Windows computers which can serve as developer workstations. We can share files via source control and Dropbox and we can access the computers from Windows, from Macs, and even mobile devices like an iPad.

Good luck!
@mkennedy

Let’s talk about cloud operating systems. This article explores the options and potential of moving entirely “To The Cloud” for developers who normally demand significant offline power from their applications (IDEs, compilers, debuggers, etc). We’ll focus on .NET / Visual Studio developers, but I’m sure you can adapt this to your technology of choice.

I’ve been fascinated with what Google is doing with Chrome OS. I think finding a way to fully “live in the cloud” has a lot of promise. However, in practice Chrome OS is entirely unappealing to me. It’s just a single browser window, maximized. Yuck. I don’t mind doing most things in the browser, but I’d like multiple non-maximized browser windows and a desktop to organize things like shortcuts. That’s OK though because Chromebooks aren’t the only option. We have decent operating systems right now that can function largely in the same way if we set them up with discipline.

What we’re going to do is take a pristine Windows 7 instance and deck it out for the cloud. Even with all the choices these days, Windows 7 is still my favorite OS. I do have a Mac, several iOS devices, and I could always install Linux. Nevertheless, it’s Windows 7 for me so that’s where we’ll start.

Creating a Fresh Windows 7 Instance – Gently

I want to start from the very beginning. I don’t want old apps I’ve already installed encouraging me to “cheat” and work outside of the cloud more than necessary. At the same time I need to be able to fire up Visual Studio and do some work with an unreliable or nonexistent Internet connection So I’m keeping my current “full” Windows system intact. We’ll install a secondary instance of Windows using the awesome Boot to VHD feature in Windows 7 Ultimate. If you haven’t heard of it, I did a video walk-through here:

   Boot to VHD Screencast:
   http://www.michaelckennedy.net/blog/2009/10/20/BootToVHDScreencast.aspx

Scott Hanselman’s write up is very helpful too.

   Less Virtual, More Machine – Windows 7 and the magic of Boot to VHD:
   http://www.hanselman.com/blog/LessVirtualMoreMachineWindows7AndTheMagicOfBootToVHD.aspx

This process takes about 15 minutes. Once you’ve got everything setup, I recommend you activate and then snapshot that VHD file (from your main OS) so you can always get back to “clean”. If you’re really living the cloud lifestyle, than paving your system is much easier than before. I always keep a secondary partition around for data files anyway so that can be shared across OSes.

Now that you’ve got a truly fresh Windows 7 copy, it’s time to install just a few apps needed to set you free (and a few you’d rather not live without).

Installing “The Cloud”

While your fresh copy of Windows 7 does have IE 8 installed, that browser is entirely not up to the task of being where you spend 90% of your time. I strongly encourage you to give the latest Chrome beta a shot as your main browser. The “application shortcuts” feature of Chrome makes it much more immersive than anything IE 9 or FireFox 4 are doing (yes, I know they have pinned sites, that’s not even close).

   Chrome Beta
http://www.google.com/intl/en/landing/chrome/beta/

Got Chrome installed and set as your default browser? Good. Don’t forget to install all your Windows updates and Windows 7 SP1 while you’re at it.

What about Visual Studio?

You may be thinking, “I *NEED* Visual Studio 2010”. Yes, we do need it for sure, but stick with me here. We are NOT installing that beast on our fresh Windows 7 system. Remember, this is your Cloud OS. I even named mine Cloud OS in my boot loader to keep that focused in my mind.

We’ll talk about Visual Studio soon.

Installing Your Web Applications

You may think that you do not need to install your web apps cloud apps. There are a few installers we’ll be running such as DropBox and ChatPast, but even the plain vanilla websites such as GMail are better if you install them. That’s why Chrome is way better than the other browsers.

Let’s suppose you’ll want to use Google Docs to work with documents, presentations, Excel worksheets and so on (you will want this!). Visit https://docs.google.com, then choose “Create application shortcut” in Chrome (see image):

This not only gives you start menu items and desktop shortcuts, but it makes your taskbar truly useful and your web apps behave as regular Windows apps.

In addition, you get the true feel of your websites being applications without all the browser toolbars and other junk around it. For example, here’s how this article (barely written) looks in Google Docs right now. Notice, how it looks like an app rather than a tab buried in a busy browser. It remembers its window size and position. In short, it’s more application-like.

What Cloud Apps Should I Install?

OK, so I hope I convinced you that treating your web apps as true applications is worthwhile. Now which ones do I install? Here are a few I recommend. You may have your own favorite sites. We’ll also need a few apps that do cloudy stuff but run MSI installers on our system.

- GMail: https://mail.google.com/ (obviously )
- Google Docs: https://docs.google.com (good Microsoft Office replacement)
- Google Calendar: https://www.google.com/calendar/
- Google Contacts: http://mail.google.com/mail/contacts/u/0/ui/ContactManager
- 37 Signals LaunchPad: https://launchpad.37signals.com (Great web apps for projects)
- Amazon Web Service: https://console.aws.amazon.com/ec2/ (You’ll see why soon)
- Twitter: http://twitter.com/
- Office Online: http://office.live.com (for when Google docs fail you)
- Pandora: http://www.pandora.com/

Here are a few proper apps that are cloud-based in some way or another that require installers.

- Skype (obviously http://www.skype.com
- DropBox: https://www.dropbox.com
- ChatPast Sync Client: https://chatpast.com/

By using GMail, Google Contacts, and Google Calendar, you obviously get portability from the web. But if you have a modern phone or tablet, chances are you can keep them 100% in sync using Google’s Exchange support. I do that for both my iPhone and iPad.

There are also a couple of light-weight apps that I just didn’t want to do without and still have a solid use in a Cloud OS. I needed a good image editing app (for writing blog posts like this) and one for screen captures. So we have

- Paint.NET: http://download.cnet.com/Paint-NET/3000-2192_4-10338146.html
- Window Clippings: http://www.windowclippings.com/

Finally, there are a couple that I use to just keep an eye on my system such as

- Process Explorer: http://technet.microsoft.com/en-us/sysinternals/bb896653

Here is the full list of *everything* I have installed on my system. Just 11 applications installed. Note that I’ve hidden some that Windows installs without asking. For example, using my Microsoft Mouse forces an install of Microsoft’s Intellipoint software when I plug it in. Similarly for the touch-pad on my laptop. To me, those don’t count so I edited them out.

Escape from Outlook

You may be thinking that because you still use your company’s email that you’ll need to install Microsoft Office and the 800-lbs gorilla that is Outlook. Chances are you will not need Outlook. I have several “regular” email accounts I have to check that don’t have reasonable web options.

You can setup your GMail account to automatically pull from your other email accounts as if GMail itself was a POP3 client like Outlook. Just go to Gmail > Settings >  Accounts and Import > and configure “Check mail using POP3” as well as “Send mail as”. Here you see my other accounts have recently been checked and received mail.

Sometimes having a “Send and Receive” button for these accounts is handy. Visit the Labs section of GMail and install the “Refresh POP accounts” add-on to make the refresh button propagate a refresh off to your other accounts as well.

While you are in there, you should turn on the keyboard shortcuts and take 5 minutes to learn the important ones. For example, / will take you to search, i to the inbox, e archives, -c composes an email in a separate window, etc.

Where Are the Developer Tools?

You probably didn’t see Visual Studio in that list, because it wasn’t there! In the next blog post, I’ll take you through setting up an Visual Studio 2010 instance in the cloud along with all the other developer tools and servers you might need such as SQL Server, and MongoDB, and LINQPad, and so on. Stay tuned.

When I have that post online, I’ll be sure to add a link here. If you just can’t wait, here’s a hint of where we’re going next.

[Update: Read Part 2 Now]

Good luck!
@mkennedy

Here’s an article about Getting Things Done and how I use Backpack from 37Signals to make it happen.

I’ve recently taken on some roles where I have a lot more loose-ends in my life and adopting Getting Things Done (GTD) has really helped manage everything. I’m also a huge fan of the 37Signals suite of products (Backpack, Basecamp, etc). That should be obvious from what we’re doing over at ChatPast with integrating instant messaging and 37Signals applications.

So using 37Signals to implement Getting Things Done (GTD) was the obvious choice for me. After looking at both Backpack and Basecamp, it seems that Backpack is the clear winner for implementing GTD. You’ll see why shortly.

There have been some articles already written about Backpack + GTD. Brett Kelly wrote a nice one over at Getting Things Done with 37signals’ Backpack – Why I’m Switching Back. However, while this was helpful in showing me that I *could* implement GTD in Backpack, it didn’t show me how to do it.

How should I organize my pages? Do I use reminders, the calendar, both or neither? There will be TODO lists, but how do I correlate them to my actual work? What about check lists? Can I have templates for repeated ones? Answering these questions is the purpose of this article.

A Step-by-step guide to implementing GTD in Backpack

In this article we’ll look at the major pieces of GTD and how to fit them into Backpack.

First, A Quick Review on GTD

I’m not going to give you full summary of GTD. If you haven’t started using GTD yet, I recommend that you read the book. Don’t like reading? Listen to Scott Hanselman interview Lane Newsom about it or watch David Allen speak about it at Google.

Here’s enough to start seeing GTD manifest itself in Backpack. First a definition, what is
Getting Things Done:

GTD is a work / life management technique which encourages you to make sure you have absolutely zero open items and actions that you are tracking in your head. You get everything that you are working on or waiting for written down. This frees your mind to be fully “in the moment” and focused on what you chose to do now as well as keeps you from forgetting anything you committed to doing.

At the heart of GTD is this decision-tree model:

[image credit Graham King]

In our GTD in Backpack world, the 8 items ringing the workflow will mostly land in Backpack. The “Do It” one just gets done and saved nowhere – although I sometimes track these just for my personal record.

GTD in Backpack – The Basic Layout

In Backpack, I have 5 pages dedicated to GTD, which is convenient because as you see, you get 5 for free. (note: if you aren’t familiar with Backpack yet, watch these
short videos before going on.)

You can see that each page corresponded to a major component of GTD. They are further partitioned inside each page – more on that below.

1. Inboxes

With inboxes, I’m not talking about just your email inbox – although that’s probably the biggest one for most people. GTD inboxes represent places that stuff that is new shows up and is waiting to be processed through the decision-tree model above. I think you’d be surprised to see how many inboxes you truly have if you took a full

account of them. GTD encourages you to do this for all sorts of things – take a full account and get them out of your mind.

That’s one of the great uses of check lists in GTD + Backpack.

2. Check Lists

A check list is any set of steps you need to remember. Sometimes it’s for one-off situations, others it’s repeating. Processing all the new stuff in your varied inboxes is a repetitive action. So I made a check list for that. Check lists, as you can see below, are TODO lists in backpack. When they are repeated, I make a template like the actual list but with the title prefix [template]. Then I make a copy to “do” the check list each time.

Here is the check list I use several times a week to make sure that I don’t forget to “process” one of my inboxes, which then feeds the other 8 types of containers ringing the workflow diagram.

You would see a very similar looking TODO list at the bottom of the [GTD] Check Lists page with the title ‘[template] Processing “in” Checklist’.

3. Next Action Lists

The Next Action Lists are pretty straight forward. As you saw above, there is a page entitled “[GTD] Next Actions”. Here I track my next action for each item or project I have open. Again the page is partition internally:

Notice that I have a personal and business one (actually several). One rule of GTD is that if your home life is a mess, it’ll affect your business life and visa versa. So manage it all here. Note also that you can add comments and attach files to the items to track additional info you might need.

If the next action came from an ongoing project (see below), then I’ve tried to put the name of the project as a prefix, for example House: in my personal items.

Finally, you’ll see that I don’t use the GTD contexts as most people might. In my world there is so much overlap for the contexts that they become meaningless. Traditional contexts might include calls, at computer, errands, at office, at home, agendas, read / review. However, I work at home with internet always on and a computer at my desk. I also do reading here. So those contexts basically all mean the same thing and thus I don’t use them.

4. Projects

As you saw in the basic structure, projects get their own page (internally partitioned). Thus I have a page entitled ‘[GTD] Projects’. This one is pretty simple. I keep track of all the ‘open-loops’ that require 2 or more actions to complete. Each one gets its own TODO list within Backpack and those lists are partitioned by work and personal.

5. Reminders

By far, the biggest GTD “bang for the buck” that you get from Backpack come in the form of reminders. However, their effect takes some exploring to see their full potential. First, let’s look at the reminders I have upcoming. Usually I have more than this, but they seem to have thinned out for the weekend.

In my reminders, you’ll see I need to finalize the home inspection details for a house I’m buying on Feb 4th. By putting it into my reminders, I can rest assured that I’ll remember it on the day that I can do something about it.

What about calendars? I make extremely heavy use of calendars for GTD. But not within Backpack. I use Google calendar, which then I sync to Outlook, my iPhone, my iPad, and I can also access via the web. If this had been an appointment, I would have skipped the reminders and just put it on my calendar. But it’s an action with a date, so it goes in the reminders.

Getting Notified of Reminders

The first place to leverage reminders is to have them show up as text messages on your phone. To do this, go to “My Info” in backpack, in the section “Mobile phone reminders”, check the checkbox for “Send text message reminders to my mobile phone”.

Next, you’ll want to see these reminders in your calendar (but not as appointments!).
That’s where the crazy URL starting with iCalendar:     webcal://domain.backpackit.com/ical/…
comes in. You can subscribe to this in Outlook or, as in my case, in Google calendar.

In my calendar, then have all these actions + dates show up alongside my day.

I think I had to replace ‘webcal:’ with ‘http:’ in order to add this to Google Calendar though.

6. Waiting-For Lists

Waiting lists are for situations where I know I’ll need to follow up on something but no necessarily by any particular date – in that case a reminder doesn’t make sense for me. But several times a week, I’ll scan the list and close those loops if they other person hasn’t responded by then. Here’s how that looks in Backpack:

Note that the dates I manually typed in are not the due dates. They are the creation dates (Backpack doesn’t track this). So I know how long I’ve been waiting.

7. Reference Material

One of the huge advantages that Basecamp has over other systems that are just simple TODO lists is the ability to attach rich reference material and additional information to the TODO item that represents a next action, or waiting item, and so on.

You can add comments to any TODO item. For basic information, this can be simple text. For example, if you needed to renew your insurance, you could attach the phone number and policy number to the TODO item so you don’t have to look it up.

For non-textual information, you can also attach files (pictures, word docs, whatever) to the TODO item. Note that file attachments are a paid feature, so you can’t use this option
if you’re on the free version of Backpack.

8. Taking it with you

One of the key tenants of GTD is that you should have your info with you nearly all the time. You might be thinking that using Backpack means you will only be able to get to it when you’re on the web.

If you have an iPhone, iPad (or even iPod touch), you can always have an offline copy available using an app called Satchel found in the Extras section.

Here’s a few screenshots from the iPhone:

Here’s a few screenshots from the iPad (click to enlarge):

9. What about Basecamp?

Some projects are either shared by a team of people or are truly massive with many steps, artifacts, milestones, and files. These projects are better managed outside of Backpack. In that case I use the paid version of Basecamp and simply add a note to the project entry page as you can see here.

Conclusion

If you’ve been thinking of using Backpack for GTD, I hope this has given you something to work with. I certainly don’t promise to be a GTD expert. But this setup works well for me and I think it will for you too.

Thanks!
@mkennedy

What I want to talk about is all the cool open source projects I found just by flipping through the pages of the NuGet directory in the Visual Studio “Add Library Package Reference” dialog.

1. RazorEngine at  http://razorengine.codeplex.com/

RazorEngine is templating engine built upon Microsoft’s Razor parsing technology. The Razor Templating Engine allows you to use Razor syntax to build robust templates No need to learn a custom clunky API for generating things like HTML and emails and so on. Just use the hot new @Razor syntax from ASP.NET MVC 3.

2. YUI Compressor for .Net at http://yuicompressor.codeplex.com/

YUI Compressor for .Net is is a .NET port of the Yahoo! UI Library’s YUI Compressor Java project. Do you have a bunch of CSS and JavaScript files and you want your page to load faster. This is a great way to do it from ASP.NET.

3. 51degrees.mobi at http://51degrees.codeplex.com/

Want to build an ASP.NET MVC website that has both a desktop and mobile version from the same project? 51degrees.mobi Foundation is an ASP.NET open source module which detects mobile devices and browsers, enhancing the information available to ASP.NET. Mobile handsets can optionally be redirected to a home page designed for mobile phones. Smart phone and feature phones are all supported.

4. Lucene.Net at  http://lucene.apache.org/lucene.net/

Lucene.Net is a source code, class-per-class, API-per-API and algorithmatic port of the Java Lucene search engine to the C# and .NET platform utilizing Microsoft .NET Framework. Want indexed full-text search from .NET? Here you go.

5. MvcMailer at  http://www.codeproject.com/KB/aspnet/MvcMailerNuGet.aspx

Send a professional looking HTML email from your ASP.NET MVC simply by pointing at a particular view.

6. DotNetZip at http://dotnetzip.codeplex.com/

DotNetZip is an easy-to-use, FAST, FREE class library and toolset for manipulating zip files or folders. Zip and Unzip is easy: with DotNetZip, .NET applications written in VB, C# – any .NET language – can easily create, read, extract, or update zip files. For Mono or MS .NET.

7. FileDB at http://filedb.codeplex.com/

Ever wanted to have a single, simple file that you can access as a basic database without the fuss? FileDB is a free, fast, lightweight C# (v3.5) DLL project to store, retrieve and delete files using a single archive file as a container on disk. It’s ideal for storing files (all kind, all sizes) without databases and keeping them organized on a single disk file.

8. Chargify.NET at http://chargify.codeplex.com/

If you need to do recurring billing, like I need to over at  http://chatpast.com, then  Chargify.NET is super handy. It’s comprehensive C# API wrapper library for accessing http://chargify.com, using XML or JSON to read/write.

9. Facebook C# SDK at  http://facebooksdk.codeplex.com/

Facebook is all the rage these days, and python, well… Here’s a .NET SDK for Facebook. The Facebook C# SDK helps .Net developers build web, desktop, Silverlight, and  Windows Phone 7 applications that integrate with Facebook.

10. OpenSearch Toolkit at  http://opensearchtoolkit.codeplex.com/

OpenSearch is a great way to “light-up” the search features of your site. For example, if you visit http://chatpast.com with Firefox and pull down the search engine selector, you’ll see “Add ChatPast Search”. Do the same in Chrome and then type chatpast.com in the omnibox and hit tab then you’re searching directly into your chat history (even when you’re not on chatpast.com). So obviously OpenSearch is awesome. This toolkit makes that easier. The OpenSearch Toolkit provides ASP.NET developers with drop-in support for OpenSearch. This lets you quickly and easily publish valid search suggestions to all the major browsers.

11. NoRM at http://normproject.org/

LINQ, NoSQL and MongoDB, what more could you want? NoRM is how you make it happen. NoRM is a .Net library for connecting to the document-oriented database, MongoDB.

11.5. Simple MVVM for WPF, SL, and WP7 at http://simplemvvmtoolkit.codeplex.com/ (added after original post, hence the .5)

Simple MVVM Toolkit makes it easier to develop Silverlight, WPF and WP7 applications using the Model-View-ViewModel design pattern. The purpose of the toolkit is to provide a simple framework and set of tools for getting up to speed quickly with Silverlight applications based on the MVVM design pattern. The emphasis is on simplicity, but it should contain everything you need to implement MVVM for real-world line of business applications.

Keep in mind, that to get the latest version of any of these projects, you can just use the Visual Studio ”Add Library Package Reference” dialog because of all the NuGet goodness. No need to download and manage these project references at a low level.

Enjoy!
@mkennedy

Thanks to everyone who came to my two sessions at the Portland CodeCamp this weekend.

The NoSQL Movement, LINQ, and MongoDB – Oh My!

Panel Discussion: NoSQL vs. RDBMS

You can download the slides, demo code, and the mongoctx code snippet for Visual Studio here:

NoSQL + MongoDB + LINQ:
http://www.michaelckennedy.com/Talks/Downloads/PortlandCodeCamp/Kennedy-PdxCodeCamp-2010-MongoDB-NoRM-LINQ.zip

NoSQL vs. RDBMS Panel:
http://www.michaelckennedy.com/Talks/Downloads/PortlandCodeCamp/Kennedy-PdxCodeCamp-2010-NoSQL-vs-RDBMS-Panel.zip

If you missed the sessions and want to see some MongoDB + LINQ or if you would like to recommend this session to a friend, I’ll be doing a webcast on a very similar talk tomorrow:

NoSQL + MongoDB + LINQ Webcast

Cheers!
@mkennedy

[Update: This webcast is in the past, but you can download the video and audio recordings.]

Download Video (WMV)
Download Audio (MP3)

I’m happy to announce I’ll be doing a free webcast in the DevelopMentor webcast series on MongoDB, .NET, LINQ, and NoRM.

NoSQL Movement, LINQ, and MongoDB 
Tuesday May 25th – 11AM PST
Register here

I hope to see you there. We’ll be building out a website in ASP.NET MVC with MongoDB as the back-end using LINQ. There will be plenty of code so it should be fun and educational.

Cheers!
@mkennedy

This article is a follow up one I wrote last week entitled “The NoSQL Movement, LINQ, and MongoDB – Oh My!”. In that article I introduced the NoSQL movement, MongoDB, and showed you how to program against it in .NET using LINQ and NoRM. You can also watch my conference presentation at MongoDB Seattle 2011 or this DevelopMentor webcast.

I highlighted two cornerstone reasons why you might ditch your SQL Server for the NoSQL world of MongoDB. Those were

1. Ease-of-use and deployment
2. Performance

For ease-of-use, you’ll want to read the original article.

This article is about the performance argument for MongoDB over SQL Server (or MySql or Oracle). In the first article, I threw out a potentially controversial graph showing MongoDB performing 100 *times* better than SQL Server for inserts.

“A potentially controversial graph showing MongoDB performing 100 times better than SQL Server”

We’ll see source code, downloadable and executable examples and you can verify all of this for yourselves. But first, here’s a new twist on an old proverb:

“Data is money”

If your application is data intensive and stores lots of data, queries lots of data, and generally lives and breathes by its data, then you’d better do that efficiently or have resources (i.e. money) to burn.

Let’s imagine you’re creating a website that is for-pay and data intensive. If you were to attempt to plan out your operating costs per user to help guide the pricing of your product then the cost of storing, querying, and managing your data will likely be a significant part of that calculation.

If there is a database that is 100 times faster than SQL Server, free, easy to administer and you program it with LINQ just as you would with SQL Server then that is a very compelling choice.

When you have such a database, it means you can run your system on commodity hardware rather than high-end servers. It means you can have fewer servers to maintain and purchase or lease. It means you can charge a lot less per user of your application and get the same revenue. Think about it.

“It means you can charge a lot less per user of your application and get the same revenue. Think about it.”

One more story before we see the statistics. Kristina Chodorow from 10Gen gave a talk a few weeks ago at San Francisco’s MySQL Meetup entitled “Dropping ACID with MongoDB”. You can watch the recording here:

http://www.ustream.tv/recorded/6146875

[The audio and video isn’t too hot, but the content is. Skip the first minute without
audio.]

During this talk, Kristina describes SourceForge’s experience moving from MySql to MongoDB. On MySql, SourceForge was reaching its limits of performance at its current user load. Using some of the easy scale-out options in MongoDB, they fully replaced MySQL and found MongoDB could handle the current user load easily. In fact, after some testing, they found their site can now handle 100 times the number of users it currently supports.

Not convinced of this NoSQL thing yet? Fair enough. Here are some graphs, some stats, and some code.

The scenario:

Model a data intensive web application aiming to support as many concurrent users
as possible. There will be users from the web application itself. But there will also be users from an API and external applications. Users will interact with the data by having nearly as many inserts as they do queries. Their inserts are all small pieces of data and are all independent of each other.

Let me just get this out of the way and I mean the following in the nicest of ways: I don’t care about your scenario or use-case. The scenario above is what I’m trying to model. I’m not trying to do bulk-inserts or loading large files into databases or anything like that. MongoDB may be great for these. SQL Server may have specialized features around your use-case, etc. They don’t apply in my scenario. So please don’t wonder why I’m not using bulk inserts or anything like that in the examples below.

Insert Speed Comparison

It’s the inserts where the differences are most obvious between MongoDB and SQL Server.

These inserts were performed by inserting 50,000 independent objects using NoRM for MongoDB and LINQ to SQL for SQL Server 2008. Here are the data models:

MongoDB basic class

SQL Server basic class

I ran five concurrent clients hammering the databases with inserts. Here’s the  screenshots for  running against MongoDB and  against SQL Server. Let’s zoom into the most important result with the output from one of five concurrent clients:

MongoDB:

SQL Server:

That’s right. It’s 2 seconds verses 3 1/2 minutes!

Now to be fair, this was using LINQ to SQL on the SQL side which is slow on the inserts. After discussing these results with some friends, I re-ran the tests using raw ADO.NET style programming and saw a 1.5x-3x performance improvement for SQL. That still leaves MongoDB 30x-50x faster than SQL.

Query Speed Comparison

Now let’s see about getting the data out using the same objects above on the indexed Id field for each database.

Here MongoDB still kicks some SQL butt with almost 3x performance. If we were to leverage the mad scale-out options that MongoDB affords then we could kick that up to many times more.

“If we were to leverage the mad scale-out options that MongoDB affords then we could kick that up to many times more.”

Complex Data and the Real World

Feel like that was an overly simplified example? Here’s some real world data with foreign keys and joins. Below is the complex data model.

MongoDB:

SQL Server:

It shouldn’t surprise you that MongoDB does even better here without its joins.

The Hardware

All of these tests were run on a Lenovo T61 on Windows 7 64-bit with a dual-core 2.8 GHz processor using the 64-bit versions of both SQL Server 2008 Standard and MongoDB 1.4.1. You can even see a picture of the computer here: http://twitpic.com/hywa8

Your Turn

If you want to see the entire set of data above as an Excel spreadsheet, you can download that here:

http://www.michaelckennedy.com/Downloads/sql-vs-mongo.xlsx

You can also download the sample code. Before you do, realize I haven’t done a bunch of work to make it super easy to run. But you should be able to figure it out. Just turn the knobs on the PerfConstants class for the number of inserts and queries. Then comment or uncomment sections of the code in the clients for your scenarios.

The expected use is that you’ll start the launcher application then use it to launch five concurrent clients at exactly the same time.

Download Sample:

http://www.michaelckennedy.com/Samples/SpeedOfSqlVsMongoDBAnddotNetSample.zip

Got feedback? Write a comment or contact me on Twitter:  @mkennedy or find me in
any of these other ways.

Thanks!

Some thanks are in order for all the help I got bouncing around ideas as well as trying different scenarios. Thanks to

Eric Cain @arcain
Jim Lehmer@dullroar
Karl Seguin @karlseguin

Cheers!
@mkennedy 

Maybe you’ve heard people talking about ditching their SQL Servers and other RDBMS entirely. There is a movement out in the software development world called the “NoSQL” movement and it’s taking the web application world by storm.

“Insanity!” you may cry, “for where will people put their data if not in a database? Flat files? Tell me we aren’t going back to flat files.”

No, but in the relational model, something does has to give. The NoSQL movement is about re-evaluating the constraints and scalability of data storage systems in the light of the way modern web applications generate and consume data.

The outcry about flat files above is meant to highlight an assumption developers often have about building data-driven applications: Data goes in the database (SQL Server, Oracle, or MySql). Just maybe, if we are really cutting-edge, we might consider storing our data in the cloud, but the choices generally stop there.

The NoSQL movement asks the question:

“Is the relational database (RDBMS) always the right tool for data storage and data access?”

Starting from an RDBMS is virtually an axiom of software development. However, those of us who are excited about NoSQL believe that relational databases are not always the answer. I think this highlights one of the reasons this NoSQL thing is called a movement. People are realizing they have a choice where they thought they had none.

The converse is, of course, also true. The NoSQL databases are also not always the right choice either. If you look carefully however, you will find that they are a good choice much of the time. Don’t take my word on it. Ask Facebook, Twitter, Digg, SourceForge, WebEx, Reddit and a bunch of other companies here and here that are using NoSQL databases.

This move towards NoSQL is driven by pressure from two angles in the web application
world:

• Ease-of-use and deployment
• Performance - especially when there are many writers as compared to the number of readers (think Twitter or Facebook).

Choosing NoSQL for Ease-of-Use and Deployment

I cover the programming model in detail as well as introduce the actual database
server below. For some vague motivation, let me just give you a quick look at how
you define the data model and maintain it.

Why, in the name of all that is right, do we have to model our system twice? Once in the database and once, in parallel, in code? With NoSQL, you have one place to do that – in your C# classes.

* You may have to run a transformation tool if you’re making radical data changes, but that’s true in SQL systems as well.

Choosing NoSQL for Performance

When the number of concurrent clients using your application – and thus your database - is reasonably small (let’s say 500 users as a baseline) RDBMS can work great. But what if that number grows? And if you are writing a web app, you definitely want that number to grow. At 50,000 users, can you still run on a single instance of SQL Server or MySql? How powerful does your hardware have to be to handle that? What about at 500,000 or 5,000,000 users, still good?

I’m sure there are some of you out there thinking, “What a minute now! There are plenty of systems with tons of users built upon relational databases.”

It’s true, there are. But how much expensive hardware and software do these require? How easy is it to leverage *commodity* hardware and free software? A basic SQL Server  cluster might run you $100,000 just to get it up and running on decent hardware. Rather than leveraging crazy scaling-up options, the NoSQL databases let you scale-out. They make this possible (dare I say easy?) by dropping the relational aspects of a database. Some NoSQL systems such as MongoDB get even better scalability by loosening some of the durability guarantees – which they backfill somewhat with redundancy (more on MongoDB shortly).

“Ok, ok. So it’s cheaper and simpler,” you say. “How much faster than the finely tune system that is SQL Server 2008 can these open source NoSQL systems be?”

The answer is: MUCH MUCH FASTER. Here’s a simple comparison of running a bunch of concurrent inserts into SQL Server 2008 and MongoDB on the same computer.

Looks like under heavy load, I’d say it’s about 100 times faster. I’m sure there going to be tons of second guessing this graph and so on. Hold your comments please! I’ll be posting a full performance comparison with source code soon. Let me just say that I think the comparison was fair – I’ll back that up in a later post.

NoSQL and a New Programming Model

If we do not have joins and primary / foreign key relationships, how do we associate related data? In NoSQL, there is a way to mimic foreign keys for certain relationships. However the main answer is that you do not disassociate your data in the first place.

I’m sure that you’ve all heard of the object-relational impedance mismatch. A large part
of that mismatch comes from the fact that we normalize the data in our database to the extreme and then use joins to reassemble that data. Not only does that cause this so-called impedance mismatch, but those joins can be really slow and they can be the death of any scale-out solution. The key to many of the NoSQL databases’ scalability is that they do not use joins. You simply save large swaths of your data as a single blob (which in MongoDB’s case, is still deeply queriable).

Shortly we’ll look at an example where we build out a disconnected, offline RSS reader that uses MongoDB and LINQ to store its data. But just think about how you might structure your data storage if you could save entire object graphs and still query them? Your “row” might be a Blog object which has an array of BlogEntries which contain the entry text, link, date, etc. Then your *entire* query to pull all the details of a single blog would hit a single “table” in the database. That might look like this query which has one result:

var blog =
       (from b in ctx.Blogs
       where b.Id == requestedBlogId
       select b).FirstOrDefault();

There are no joins or anything like that because you’re saving objects not columns and those objects contain their collections already (e.g. RssEntries). There is an important distinction to make here. These NoSQL databases generally are *not* the same as object databases. They are what are known as document databases. There’s actually a
big difference between the two.

Introducing MongoDB

The NoSQL database we are using in this example is MongoDB. This is free, open-source database which runs on Windows, Linux, and Mac OS X systems. You can access it from many platforms including .NET, Ruby, Java, PHP, and so on.

We’ll be using .NET and C# of course. You have several options when choosing how to access MongoDB from .NET but generally that means using LINQ and a light-weight object-mapper on top of MongoDB itself. Note that common terminology might categorize the object mapper that moves objects into and out of the database as an ORM. While that’s OK, there is technically no “R” in this ORM because MongoDB is not relational. Hence I’m calling simply an Object-Mapper (OM).

In MongoDB nomenclature, theses libraries are called drivers. My favorite .NET driver
is called NoRM. It’s being actively developed and was created by Karl Seguin, Andrew Theken, Rob Conery, James Avery, and Jason Alexander. You can find NoRM on GitHub and discuss it in its related Google Group.

If you want to learn more about MongoDB you should listen to these Podcast interviews:

Michael Dirolf also has a great book in the works. You can catch a preview of it on
Safari Books Online. Here’s the amazon page:

MongoDB: The Definitive Guide.

NoSQL in Action

Let’s write some code. The first step typically in a data-driven application is to spec out the database. Then we’d use LINQ to SQL or Entity Framework to generate the ORM classes. MongoDB is different. MongoDB has no schema or rather its schema is flexible and defined via usage rather than being predefined in the database. So our first step is to define the classes we’d be storing in the DB via NoRM.

We’re going to define 3 classes: Blog, RssEntry, and RssDetail. The Blog object will contain a collection of RssEntry objects. In practice you might just go with the Blog and RssEntry classes. But I wanted to model both the embedded case (Blog + RssEntry) and the loosely defined foreign key style relationship that mimic joins (RssEntry + RssDetail). That way we can demonstrate both use-cases.

Here’s a taste of the Blog class:

public class Blog
{
	public ObjectId _id { get; set; }
	public string Name { get; set; }
	public string Url { get; set; }
	public string RssUrl { get; set; }
	public List Entries { get; set; }
      // ...
}

Notice that it contains a collection (Listreally) of RssEntry objects. That’s the relationship supported by nesting. The Blog class just has this collection as part of its data model.

The RssEntry class has the summary info for a blog entry:

public class RssEntry
{
	public ObjectId _id { get; set; }

	public Guid UniqueId { get; set; }
	public DateTime PostedDate { get; set; }
	public string Title { get; set; }
	public string RssGuid { get; set; }
}

And the larger data is stored in the RssDetails class (for example the text of the post):

public class RssDetails
{
	public ObjectId _id { get; set; }

	// this is kinda like the foreign key.
	public Guid RssEntryId { get; set; }

	public List Categories { get; set; }
	public string Link { get; set; }
	public string Text { get; set; }
	// ...
}

Let’s see how we insert an entire set of Blog data into the database. We begin by generating the objects (Blog, RssEntry, etc) in memory and then serializing them via NoRM to MongoDB much as you would in LINQ to SQL. The difference is this will actually generate the collections (analogous to tables) if they don’t already exist and it will define the implicit schema to match our objects:

void SaveBlogToMongoDb(
	string rssUrl, XElement root, RssDataContext ctx)
{
	Blog blog = new Blog();
	blog.RssUrl = rssUrl;
	blog.Name = GetBlogName(root);
	blog.Url = GetBlogUrl(root);

	blog.Entries = ParseEntries(root);
	IEnumerable details
		= GetDetails(blog.Entries, root);

	foreach (RssDetails detail in details)
	{
		ctx.Add(detail);
	}

	ctx.Add(blog);
}

Here we are using a class called RssDataContext which we wrote manually. It is very similar to what LINQ to SQL and Entity Framework use to do the object-relational mapping. Want to do a query? Do you know LINQ? Well then you’re all set:

var results =
    from b in ctx.Blog
    where b.Name.Contains( "MongoDB" )
    select b;

How do you add a new entry to an existing blog and update it in the database?

void AddEntry(Blog blog, RssEntry entry)
{
	blog.Entries.Add(entry);
	ctx.Save(blog);
}

We leverage the fact that the blog.Entries collection is a List and just add to it. Then save will update the record in the DB.

All this works great and is highly performant. But do be careful as not all the LINQ operations are fully implemented yet in NoRM and some (like join) may never be added because MongoDB doesn’t support it.

To get started, download MongoDB the tools and server here:

http://www.mongodb.org

You unzip the zip file and run the mongod.exe program. Be sure that you have created the C:\data\db folder. It appears at first that you have to run MongoDB in a console window. But you can register it as a Windows Service:

Here’s some helpful advice on installing MongoDB as a Windows Service (there is
a small bug you have to work around):

http://www.deltasdevelopers.com/post/Running-MongoDB-as-a-Windows-Service.aspx

There’s also a management console (and I mean “console”):

It’s a little different. You’ll get used to it. The means of interaction with the server is through JavaScript rather than T-SQL and the storage format is a binary form of JSON as you can see.

For a project I’m working on I’ve built a Windows Forms UI that lets me manage the database easily by just adding an object data source and doing some drag-drop magic in Visual Studio. Generally I look down upon that sort of development, but for an admin tool it’s just fine.

Now It’s Your Turn!

Try it out for yourself. Download MongoDB and the NoRM driver and build some apps. You may also want to check out the source code for my demo app:

Download Sample: RssMongoSample-Kennedy.zip

Got feedback? Write a comment or contact me on Twitter where I’m @mkennedy or find me in any of these other ways.

Recommended Reading:

Here are some other blogs on this subject.

I came across what I think is an awesome technique for seeing how your web page will look as you edit it. This is WAY beyond WYSIWIG:

1. Load the page you’re working on in ALL the browser you care about. I’m using Chrome 4, FireFox 3.6, and IE 8.
2. If you have the monitor space, cascade these browsers side-by-side.
3. Add a meta-refresh tag to the header of that HTML file you’re working on (or which consumes the CSS you’re building)
4. Now here’s the sweet part:Edit the page in Visual Studio, notepad, whatever. When you press save all  Browsers reload their view in a few seconds!
5. Now you get real WYSIWIG on real browsers.

That’s it. The technique is totally low tech and would have worked for years. But I found it really helpful. Hope you do too.

Be sure to keep watching here. I promise a cool site will be announced soon!

Cheers!
Michael
@mkennedy

First, briefly why does one care about EC2? Well maybe you are buying into the whole cloud computing story which lets you cheaply out-source your computer hardware for amazingly cheap prices (staring around $0.20 / hour for a dedicated machine). That’s a great reason and Microsoft and Google have interesting plays there too.

Personally I just want a simpler way to create virtual machines. We’ll have full admin access over remote desktop to our system to install whatever we want. I’m putting Visual Studio 2010 Beta on mine to play around with that software without ’polluting’ my real system.

Here we go. If you don’t delay I suspect this would take you about 20 minutes from start to login! Subsequent virtual machines are much faster to create and launch because the can be based on pre-configured images.

1. Create an Account

Register for an Amazon Web Services account at http://aws.amazon.com/.

2. Enable EC2 Features

Enable Elastic Compute Cloud for your AWS account at http://aws.amazon.com/ec2/.

3. Launch a New Instance

Use the AWS Management Console to launch and manage your virtual images: http://aws.amazon.com/ec2/home. As the console says, choose “Launch Instance” under the “Getting Started” section. You will be presented with a list of pre-configured images. We’ll start with a stock Amazon Windows 2008 server image.

4. Choose a Base Image

Now you’ll be presented with a list of pre-configured virtual disk images. This time we’ll setup a 64-bit Windows 2008 Server (Data Center Edition). Just choose “select” out of the list below:

5. Use the Request Instances Wizard

Use the Request Instances Wizard to configure the newly created instance which includes configuring the security, choosing an encryption key, opening ports in the firewall, and kicking off the new instance. Below you’ll see the encryption key step – be sure to download the key pair as you’ll need it for retrieving the administrator password.

6. Launch!

Here’s what you can expect for the review screen of the Request Instances Wizard. Press launch and you’re almost there.

7. Launching… (AKA Wait 5 Minutes)

After you launch you’re instance you’ll get a confirmation screen to
show you it’s being prepared and allow you to configure durable storage
and IP addresses (both entirely optional).

8. Back to the Management Console

Now if you choose “View your instances…” you’ll see that your instance is being prepared – it has a yellow pending status. This screen doesn’t always refresh on its own so use the refresh button in the upper right of the console (rather than your browser’s refresh button).

9. Running!

After a few minutes your instance with the yellow icon will turn green and be in the running state. Note that at first this really means booting up so you can’t get to it right away. Give it another minute or two…

10. Login Part 1: Getting the Credentials

Now you’ll want to login. Of course, the system was created with an administrator account which has a strong password. You’ll need to retrieve that password using the “Instance Actions -> Get Windows Admin Password” option.

11. Login Part 2: A Little Hasty

You’re probably excited to get this thing running and if you try right away you’ll get another message telling you to be patient and try again in a few minutes. Just keep trying.

12. Login Part 3: Using Private Key

Eventual the new system is up and running and you can get the password. The first step here is to pass in your encryption key from the wizard step before.

13. Login Part 4: Administrator Account and Password

Pass in the encryption keys and you’ll see the username and password (don’t get excited, I already changed the password!).

14. Login Part 5: Finding the Machine Address

When your instance starts, it’ll be given an Internet visible DNS name that you can use to connect via Remote Desktop. You’ll find it in several places. One of them is highlighted below. Note that this address changes as you start and stop your instance.

15. Connected!

Now just fire up Remote Desktop, use the Administrator account and password from step 13 to log in. Now you have full access to your Windows 2008 machine. You can do with it what you will, install software, start serving web pages, etc.

16. A Word of Caution

If your intent is to run a web server, then let it run. But if you are just using this for your own purposes and don’t need it when you’re not logged in to the machine, be sure to return to the Management Console and stop the instance. You can alternatively do that by choosing “Shutdown” instead of logging out of your Remote Desktop instance.

I hope you found this walk-through helpful. I just learned most of this myself so I figured I’d blog it and everyone can learn from it.

Cheers!
Michael

Azure cloud on horizon: The devil is in the data architecture details

Here’s a small excerpt. If you’re interested in Windows Azure and Cloud Computing, read on…

Microsoft did a good job when they designed Azure, according to Kennedy. “The company encourages you to build scalable reliable systems by basically making it really hard to do the stuff that makes systems unreliable,” he said.

There are many developers curious about cloud computing, but most are being rather cautious. Directions on Microsoft’s Sanfilippo said he’s talked to more developers that are concerned about building on top of their existing work than re-coding everything to work in the cloud.

“There’s still an education bit that has to happen about what kind of applications are appropriate for Azure. But I think there’s a lot of curiosity about Azure,” Kennedy said.

Still, he continued, “I don’t know many projects that are betting the bank on Azure yet.”

Note: This is a little dated as it was publish in July 2009 – some how I missed the original publication – but it’s still an interesting read.

Thanks Rob and Jack for the article and conversation.

10 Features in .NET 4.0 that made Me Smile

Read it on the DevelopMentor website: http://www.develop.com/tenfeaturesdotnet4

I am republishing it below for you all to enjoy on your RSS readers.

Cheers,
Michael

10 Features in .NET 4.0 that made Me Smile

I have been reviewing some of our upcoming classes at DevelopMentor this week. One of those classes, What’s New in .NET 4.0, left me excited for things to come. There are a bunch of small but wonderful features discussed in that class. I thought I’d take this opportunity to write a few of them up and share the joy. I bet some of them make you smile too.

1. The Parallel Extensions for The .NET Framework will be built into mscorlib.dll.
    The fact that PFx will be part of the core .NET library says a lot about how much faith and support it’s getting within Microsoft. BTW, here are some really great demos for PFx in .NET 4.0.
2. PFx introduces a new threading construct: Barrier.
    Barrier lets you define rendezvous points in your code where multiple concurrent  operations can automatically sync-up. Here’s an example.
3. Code contracts.
   Code contracts allow you to assert truths about your code as if you are writing a unit test. But these assertions live within your production code and are both verified by the compiler as well as the runtime. Here’s the original research project that lead to this feature on Microsoft Research.
4. The WPF and Silverlight designers mostly work.Now this shouldn’t be a point to make me smile or get excited about, but it is. The pain and suffering around the Visual Studio support for WPF and Silverlight designers has been so bad that a mostly-working, and sometimes truly innovative design-time experience within Visual Studio gives me real hope for these technologies. I’m actually excited about them now.
5. Support for the MVVM pattern across both WPF and Silverlight.
   Speaking of that XAML stuff, if you write WPF or Silverlight code and don’t know MVVM, stop reading this article and learn about it here. I’ll wait. Ok, now you too should be excited to hear that there is improved support for MVVM across Silverlight and WPF in a unified way. Smiles baby!
6. WF (Windows Workflow Foundation) has an AsyncCodeActivity class.
   While WF has traditionally supported both synchronous (activities that execute immediately) and asynchronous activities (for long running senarios where the workflow becomes idle and is unloaded from memory [click here for more details]),  there has been an unserved middle ground. If you want to use threading in your activity and allow the workflow to go idle without it being unloaded from memory you were basically out of luck. This is the problem solved by the AsyncCodeActivity. WF 4 now has a class which has a BeginExecute / EndExecute pair of methods which much more closely models the regular .NET async design patterns.
7. WF has a rehostable designer (really, they mean it this time).
   There are some great uses for giving regular users a WF designer experience with the right granularity of activities. Now it’s much easier. Here’s an app that rehosts the designer:
8.  Configuration-free WCF Hosting.
   Hosting WCF services is now like hosting ASMX web services if you like the defaults. Just throw out a service + contract + address and it’s up and running. That’ll save a bunch of configuration goo. Smiles!
9. No more *.svc in our RESTful urls in IIS.
   With the ASP.NET routing framework and WCF REST introduced in .NET 3.5, we can create beautiful, expressive Uri’s for our websites. For example:http://lookatthiswith.me/watch/introBut this falls apart with WCF REST when we host it in IIS. Our service Uri’s look like this:
   
   http://lookatthiswith.me/services/lookieservice.svc/lookup/json/cf7And now we have this ugly .svc part-way through our Uri! Ick. Well, in .NET 4 that
   Uri is much more customizable and the .svc is gone. Smiles!
10. ASP.NET MVC has wicked JavaScript support.
    JQuery is there by default. That’s awesome. But there is also a class similar to the Html class (for HTML helpers) called Ajax. This static class has functions like Ajax.ActionLink and effectively brings the functionality of UpdatePanel to MVC!

Well there you have it. 10 awesome things in .NET 4 that made me smile this week. I hope you find some to be welcome additions yourself! If you want to learn more about .NET 4.0,  check out our recorded webcasts here: http://www.develop.com/dotnet4webcasts. Also have a look at my article from last month Six Things That’ll Surprise You About .NET 4.0. Finally, if you have some training funds laying around, I’d love to spend a week talking about these ideas with you in our What’s New in .NET 4.0 What’s New in .NET 4.0 class.

Michael Kennedy is an instructor for DevelopMentor where he specializes in core .NET technologies as well as agile and TDD development methodologies. Keep up with Michael via his Web site and blog at http://www.michaelckennedy.net or on Twitter: @mkennedy.

Here’s all the related downloads. We had some microphone troubles so I want to  apologize in advance for the sub-optimal sound quality.

You can also download the slides and peepleocity.com sample website built during the presentation.

Join me Monday, November 23rd and register here:

http://bit.ly/aspwebforms

Note: This event is in the past, but you can watch the recording here:

We’ll talk about integrating ASP.NET’s routing infrastructure into existing an ASP.NET WebForms application. This allows you to build SEO websites with URLs like

http://dotnet.ubbuzz.com/tag/.NET+4.0

while still taking advantage of all the productivity features of WebForms such as post-backs, controls, UpdatePanel, and so on.

We have room for a couple hundred more attendees so please register and be part of the fun. I promise lots of demos and somedisdainful comments about PowerPoint!

Share it with your friends (social, virtual, real, and other types) using the widgets below!

ASP.NET MVC: What’s that, you’d rather hear about ASP.NET MVC, not this creaky old WebForms stuff? That’s Brock Allen’s talk: http://bit.ly/intromvc

WF 4: Is WF 4 and visual programming your thing? Check out Maurice de Beijer’s WF 4 talk: http://bit.ly/meetwf4

New Parallel Extensions your thing: Check out Andy Clymer’s PFX talk. (link to follow soon).

Cheers,
Michael

Be sure to check out the write-up we did afterward where we talked about the tools and gave you a chance to try it for yourself:

TDD Space Invaders Write-up

You can also watch two other, higher level agile webcasts by Bill Nazzaro here:

Agile Webcasts at DevelopMentor

Cheers!
Michael

[Update: Get the videos and additional downloads for this webcast.]

As a follow-up to our “Avoiding 5 Common Pitfalls in Unit Testing” article we did a webcast where we took a problem from the audience and solved it live and unrehearsed on stage. These kinds of performances are always a risk but that’s part of what makes them fun.

Of course, the question is could we have done it better? Here’s your chance to try it for yourself (details below).

The Problem:

Our viewers chose to have us build the game Space Invaders. The first thing we had to do to sketch out a basic scenario we could implement. We started with a picture to remind what Space Invaders even was:

This was too big of a scenario for us to tackle in the allotted 40 minutes for programming. So then we started by creating a simpler scenario which we sketched out on the “whiteboard”:

Click for full size image.

Flushing Out the Scenario:

In doing this, a couple of things were revealed about the game.

First, we wanted to make the tank and aliens all be the same size so we could put them on a grid. But then we saw that our bullet wouldn’t fit that story, so we introduced the idea of relative sizes. We also realized that even though we drew the block, it was too complex for the first scenario and it would have to wait.

Notice that as we started writing the scenario in English, there are mistakes, irrelevancies, and problems with the order. This is OK. The thing to remember is that all of this was done for the sole purpose of creating a recipe for a scenario we could test. That scenario is the following:

[TestMethod]
public void TestSimpleKill()
{
   // 1. Create a 15x10 board.
   // 2. Place a 3x2 tank at 1x8.
   // 3. Place a 2x2 alien at 7x3 heading west.
   // 4. tank shoots
   // 5. advance 4 turns
   // 6. not won
   // 7. advance
   // 8. win
}

Now that we had the recipe, we could go about writing the code.

Here’s your chance to play at home!

1. Set your timer to 40 minutes.
2. Create a new test project.
3. Paste that method above.
4. Translate the comments into code.

If you believe there’s a better process, we invite you to try that as well.

We made it to step 4 during our presentation (download code below) and estimate another 15 minutes would have had the whole scenario done, tested, and well-factored.

Stories vs. Requirements (stories win):

We’d like to point out a couple of things about the story. First, it was quick to write the story. We did it in 5 minutes. Second, it translates well to code because it has behavior and objects working together. Let’s compare that to the requirements that this story flushed-out.

Now we want to point out that this requirements doc is much hard to understand than our story. For example, if you were to add more requirements (e.g. an alien also shoots) is that easy to determine whether we have complete requirements? It also takes much more effort to create and especially to tell if it is complete. People aren’t made to handle requirement documents well but we are story-telling machines. We embrace this in our coding techniques.

We’d also like to mention some of the tools discussed at the end.

For remote collaboration we use:

Skype (audio / video)
VNC for screen keyboard sharing
RDP (windows remote desktop) — requires Windows 2003/2008 server for pairing.

Source Control:

TortoiseSVN
TortoiseGit

Developer Tools:

Resharper
CodeRush

Testing Tools:

MsTest (in Visual Studio Professional and up)
NUnit
NCover
TortioseDiff
Approvals Tests
Approvals Tests CodeRush add-in
Rhino Mock
TypeMock

If you try this scenario yourself, please leave a comment about your experience.

Download the code and slides from the webcast here:

Code: TddWithLlewellynAndMichael.zip
   Slides dmtdd.pdf

Cheers — Michael and Llewellyn

Tuesday, October 27, 2009 at 10am Pacific time Llewellyn Falco and I will be giving a live, unscripted, and no safety-net demonstration of Test Driven Development (TDD) as part of the DevelopMentor webinar series (this particular series is a 3-part series on Agile development).

We already have a bunch of attendees registered. But we have room for as many of you who are interested in agile and TDD. Sign up here:

http://bit.ly/dm-tdd-m-and-l

In addition to core TDD techniques, you will see how an amazing technique and set of tools designed by Llewellyn called Approval Tests makes writing tests as simple as verifying an image or text file. Tired of writing 50 lines of test code for every 50 lines of production code but you still want the power of TDD? You need to learn more about Approvals and we’ll demo that live tomorrow!

I hope to see you all online. Feel free to help me get the word out by tweeting this or shouting it (see icons below).

Cheers, Michael.

Have you heard of the new feature in Windows 7 and Server 2008 R2 called Boot to VHD? It is amazing! But it’s one of those concepts that people hear about and think “hmm, interesting.” But when people see it in action it’s “OMG, I must have this!”

I recently had that experience myself and enough people asked me about it that I decided to do a quick (15 min) screencast how to setup a native boot to virtual hard drives.

If you want a great overview and step-by-step instructions, check out Scott Hanselman’s blog post:

Less Virtual, More Machine – Windows 7 and the magic of Boot to VHD

If you want to see boot to VHD in action, then check out the video here:

Download Screencast: BootToVhdKennedyWalkthrough.zip (158 MB)

http://www.develop.com/twitternetapps

I’ve republished here for my readers. Enjoy!

Building a Twitter Application in .NET

by Michael Kennedy (@mkennedy)
http://www.michaelckennedy.net

Twitter has become one of the web’s hottest properties. It is a central part of mainstream news programs such as CNN’s Anderson Cooper 360, congressional debates, and talk shows. In fact, it grew at a rate of 1400% this past year [bit.ly/jG9BG].

If your company wants to interact with your customers in a modern and engaging experience, you need to be on Twitter. In fact, if you have customers that really like or dislike you, they are probably talking about you on Twitter. You should be part of that conversation.

In this article, we will explore how to build a rich interactive experience on Twitter that goes beyond just creating a new Twitter account. We will build a .NET application that uses the Twitter API (a free service) alongside other cool technologies such as the WCF REST Starter Kit [http://bit.ly/v8mBb] and LINQ to fully leverage the Twitter experience.

Whether you want to build a community around your brand or you want to build the next real-time, social community website like .NET Developer Buzz [http://dotnet.ubbuzz.com/], this article will cover the technologies required to get the job done.

If you want to download the sample application to follow along, you can get it here:

http://www.michaelckennedy.com/Samples/SampleStatusUpdater.zip

While you’re at it, be sure to follow DevelopMentor on Twitter:

http://twitter.com/DevelopMentor_

Let’s Start Small

I will show you how to fully leverage the Twitter API, but many tasks can be accomplished using simpler tools and you may be better starting there. Let’s look at a few things that we can do without the Twitter API.

Case 1. You want to display your latest Twitter messages on your website.

Your tweets[1], as they’re called, can be consumed as a simple RSS feed. So you may want to simply pull this feed into your website rather than digging into the Twitter API and consuming custom XML or JSON formats. An example of consuming Twitter in this fashion can be found on my website’s front page [http://www.michaelckennedy.net/].

To get your RSS feed, just visit your profile page and get the RSS tweet link, e.g. ”RSS feed of mkennedy’s tweets” [http://bit.ly/guhZU]

[1] Tweet – these are what the individual messages sent on  Twitter are called. If the name sounds weird, I’m sure you’ll get used to it. Remember that there was a time when Google was just a noun.

Case 2. You want to watch and manage multiple accounts at the same time.

Most Twitter clients only support a single user. But there are a couple of good
tools that allow you to manage multiple accounts. My current favorite is one called
bDule and you can get for free at

http://www.sobees.com/bdule.

Beyond the Small

There are times when you want to do more than simply syndicate your Twitter stream. Let me give you an example. At DevelopMentor, we have had many instructors on Twitter talking about their own interests. But we didn’t have a corporate Twitter presence. We decided to create our corporate presence by pulling all our instructors individual tweets and rebroadcasting them from our DevelopMentor Twitter account:  @DevelopMentor_

We wanted to keep a sense of the original instructor who wrote the message, so we happend on an attribution. For example:

“Software Transactional Memory is released! (via @mkennedy)

and we wanted to do this in a flexible way. In short, we needed more functionality than Twitter provides.

There are actually three services that do this sort of thing and they looked promising.

http://www.connecttweet.com/
http://www.grouptweet.com/
http://cotweet.com/

But in the end, nothing completely matched our requirements. So we decided to write our own application to publish everyone’s tweets under the DevelopMentor banner. There are few simple steps involved as well as a lot of details we won’t go into yet.

1.  Take a list of Twitter accounts and download everyone’s statuses.
2. Determine which messages we haven’t seen before.
3.  Publish these new statuses under our corporate account.

We can actually implement a simplistic version of this by continuing to use the public RSS feeds of the individual accounts in conjunction with a very handy Twitter API wrapper called TwitterooCore which you can find at http://rareedge.com/twitteroo/blog/.

The Twitter API

In practice, there is simply data missing from the RSS feed that we require as well as features missing from the Twitteroo Core that move us deeper into the Twitter API. One thing you may well miss is the ability to tie together conversations. For example if Bob says “hello” and Jerry says in reply “@bob Back at you!” Twitter tracks that Jerry replied to Bob and publishes this link in the stream. To get access to these types of features and many other optimizations, you’ll need to use the REST-based Twitter API.

The Twitter API is documented at  http://apiwiki.Twitter.com/

There you can do things like get the users tweets, if their tweets are public, by requesting:

http://twitter.com/statuses/user_timeline.xml?screen_name=mkennedy

What you get back to dependent on the requested format. Here we’re asking for XML (user_timeline.xml) but we can also get JSON, RSS, or ATOM.

Similarly, we can update our status by making a POST request to:

http://twitter.com/statuses/update.xml

Here again we have the four possible formats: XML, JSON, RSS, ATOM. However this

time we’re using a form post for the request to update the user status which is then in-turn returned as XML.

Great, we have this cool REST API based on loosely-typed GETs and POSTs. Should we program against it using fundamental .NET types such as WebRequest, WebClient, and similar classes? We could. But WebRequest is so .NET 1.0 (circa 2001). There is a much newer API on the verge of release from Microsoft: The REST Starter Kit [http://bit.ly/1IF3Ji].

While this toolkit is generally geared towards building to RESTful WCF services, there’s also a great set of classes for building REST clients. We will use these classes to write our application. Let’s take a look at how we can use the Twitter API to write are simple application.

The REST Starter Kit

Part 1 – Getting the users of tweets.

We need to download the users messages as XML and convert them to .NET objects that we can consume our application. This is really straightforward because the REST Starter Kit as a cool feature to Visual Studio: “Paste XML as Types”. This feature will take an XML file and auto-generate types based on the inferred XML schema. In our case, the XML file we will use is returned from the user timeline.

http://twitter.com/statuses/user_timeline.xml?screen_name=mkennedy

Open a link in a web browser, choose view source, and copy some XML. Then go to Visual Studio, choose edit, Paste XML as Types. Of course you must have the REST Starter Kit installed for this to work.[2]

[2] In my experience I ran into some errors deserializing the response from Twitter. See the The Real World Intrudes section at the end of the article if you run into difficulties.

After generating our status related types, we can use the HttpClient class to download the statuses.

Listing 1.

private static void GetStatuses(IEnumerable userNames)
{
    XmlSerializer serl = new XmlSerializer(typeof (statuses));
    serl.UnknownElement += delegate { };

    foreach (string name in userNames)
    {
        string url = string.Format(
            "http://twitter.com/statuses/user_timeline.xml?screen_name={0}",
            name);

        HttpClient client = new HttpClient(url);

        HttpResponseMessage response = client.Get();
        response.EnsureStatusIsSuccessful();

        string contents = response.Content.ReadAsString();
        Stream stream = new MemoryStream(Encoding.UTF8.GetBytes(contents));
        statuses userStatuses = (statuses) serl.Deserialize(stream);

        if ( userStatuses.status.Length > 0 )
        {
            Console.WriteLine( "@{0}'s latest tweet: {1}", name,userStatuses.status[0].text );
        }
        Console.WriteLine();
    }
}

There are two main things happening in this code. We’re using the HttpClient class to download the web content associated with the given user’s timeline. Then we’re using the XmlSerializer in conjunction with the auto-generated XML-serializable types from the “Paste XML as Types” command. That’s all there is to it. The REST Starter Kit does most of the work for us.

Part 2 – Checking for New Messages

Now that we have all the statuses, we need to find the ones that we haven’t broadcasted from our main account and send them along to step 3. We won’t go into detail on how to track that. But you can imagine a simple database that facilitates a LINQ to SQL query like this:

Listing 2.

public StatusUpdate[] FindRebroadcastableStatuses(StatusUpdate[] updates)
{
    return
        (from up in updates
         let neverPosted = up.User.LastPostBroadcasted == null
         let afterPostDate = up.Time > up.User.LastPostBroadcasted
         where neverPosted || afterPostDate
         select up).ToArray();
}

Part 3 – Publishing the Statuses

Now that we’ve gathered the statuses of our various users, it’s time to rebroadcast them to our community. We’re going back to the Twitter API to update our status. Again will use the HttpClient class and following the RESTful principle of using POST to add new items to a given URI. We will do an HTTP POST to our status to add a new message to the account.

Listing 3.

private void UpdateStatus(string newStatus, long? replyToId)
{
    string url = "https://twitter.com/statuses/update.xml";

    HttpClient client = new HttpClient( url );
    client.TransportSettings.Credentials =
        new NetworkCredential( twitterUser, password );

    HttpUrlEncodedForm form = new HttpUrlEncodedForm();
    form.Add( "status", newStatus );
    if ( replyToId != null )
    {
        form.Add( "in_reply_to_status_id", replyToId.ToString() );
    }

    HttpContent content = form.CreateHttpContent();

    HttpResponseMessage message = client.Post( "", content );
    message.EnsureStatusIsSuccessful();
}

This time we create an HTML form using the HttpUrlEncodedForm class. We set the status field to our new status. If this status as a response to a previous tweet then we add the in_reply_to_status_id so Twitter knows to add a “in reply to…” tag to the tweet.

And there you have it. Working with the Twitter REST API is really quite straightforward. If you use the REST Starter Kit it’s downright easy.

The Real World Intrudes

But wait a minute. This is reality and building bulletproof applications is never that simple. There are at least five significant gotchas you have to address in practice when working with Twitter and the REST Starter Kit.

1. There will be times when Twitter is unavailable. You have to be ready for crashes and other types of randomness.

Twitter is one of those sites that can barely handle the traffic it is receiving. With its 1400% growth, this isn’t getting much better. So you must program defensively and assume that many of your API requests will fail.

2. The Twitter API is a free service. By default, you are limited to a small number of requests per hour. Many of the limits are around 150 API calls / hour. You may need to carefully design your application to work within the limits. Some applications simply need more data than this permits. For example, .NET Dev Buzz [http://dotnet.ubbuzz.com/] has to track thousands of users. In that case, you will have to get your application white listed with Twitter. You can do that here:

http://twitter.com/help/request_whitelisting

3. The date-time format used by the Twitter API is not directly parseable in .NET. The format returned is in the format “Fri Feb 01 18:18:08 +0000 2008″. But if we change this to “Fri Feb 01 +0000 2008 18:18:08″ it is parsable. So you might need to adjust these date-time values.

4. You will get 417 error codes when you try to talk to Twitter using the default configuration. The fix is not immediately apparent, but it is very simple. So if you see the following error:

System.ArgumentOutOfRangeException: ExpectationFailed (417) is not one of the following: OK (200), Created (201), Accepted (202), NonAuthoritativeInformation (203), NoContent (204), ResetContent (205), PartialContent (206) at Microsoft.Http.HttpMessageExtensions.EnsuRESTatusIs()

Just to set the following property: ServicePointManager.Expect100Continue = false;

5. “Paste XML as Types” doesn’t entirely work. For some reason, certain messages from Twitter are not deserializable to the types generated with this command. My experience was that I didn’t actually care about the data causing the problem. So I just removed that part of the generated type. You may have to subscribe to the XmlSerializer error events to prevent exceptions.

Conclusions

I hope you now have a greater appreciation for what you can do with Twitter and how it can help you build your brand or build engaging applications. We’ve used the REST Starter Kit to make it easy to consume the Twitter API. You’ve even seen some of the odd things that can go wrong and how to fix them. Don’t forget to download the sample application here:

http://www.michaelckennedy.com/Samples/SampleStatusUpdater.zip

Now get out there and build something cool.

Llewellyn Falco and I recently wrote an article for DevelopMentor’s Developments newsletter entitled Avoiding 5 Common Pitfalls in Unit Testing.

You can read it at the DevelopMentor website:

http://www.develop.com/testingpitfalls

I’ve republished here for my readers. Enjoy!

[Update: We have also done a webcast demonstrating some of these ideas, which we wrote up here:

http://www.michaelckennedy.net/blog/2009/10/28/TDDInvadesSpaceInvaders.aspx]

[TestFixture]
public class AccountTests
{
    private MockMailServer mockMailServer = new MockMailServer();

    [Test]
    public void WhoAreYouReceivingTheMostEmailFrom()
    {
        User you = User.CreateNew( "you" );
        User mike = SendEmailHelper( CreateUser( "mike" ), you, 3 );
        User mary = SendEmailHelper( CreateUser( "mary" ), you, 4 );
        User joan = SendEmailHelper( CreateUser( "joan" ), you, 2 );

        Assert.AreEqual( mary, you.GetGreatestEmailer() );
    }

    private User SendEmailHelper(User from, User to, int quantity)
    {
        for ( int i = 0; i < quantity; i++ )
        {
            EMail mail = new EMail()
            {
                To = to,
                From = from,
                Body = "Sample",
                Subject = "test"
            };

            mail.SetFormat( Formats.Html );
            mockMailServer.Send( mail );
        }
        return from;
     }

[Test]
  public void WhoAreYouReceivingTheMostEmailFrom()
  {
      User you = User.CreateNew("you");
      User mike = CreateUser("mike");
      for (int i = 0; i < 3; i++)
      {
          EMail mail = new EMail{To = you,From = mike, Body = "Sample", Subject = "test"};
          mail.SetFormat(Formats.Html);
          mockMailServer.Send(mail);
      }

      User mary = CreateUser("mary");
      for (int i = 0; i < 4; i++)
      {
          EMail mail1 = new EMail{To = you,From = mary,Body = "Sample",Subject = "test"};
          mail1.SetFormat(Formats.Html);
          mockMailServer.Send(mail1);
      }

      User joan = CreateUser("joan");
      for (int i = 0; i < 6; i++)
      {
          EMail mail2 = new EMail {To = you, From = joan,Body = "Sample",Subject = "test"};
          mail2.SetFormat(Formats.Html);
          mockMailServer.Send(mail2);
      }

      Assert.AreEqual(mary, you.GetGreatestEmailer());
      }

[Test]
  public void TestGatewayCallSuccessful()
  {
      var gateway = new Gateway {Mask = "ExampleCode.*"};
      var enviroment = new Dictionary();
      enviroment.Add("path", "ExampleCode.HelloWorld");
      string result = gateway.ExecuteRequest(enviroment);
      Assert.IsTrue(result.Contains("Hello World"));
  }

  [Test]
  public void TestGatewayBlocksInvalidMasks()
  {
      Assert.IsFalse(Gateway.IsValidForMask(
          "Example.*", "ExampleCode.HelloWorld"));

      Assert.IsFalse(Gateway.IsValidForMask(
              "ExampleCode.*.Extras.*", "ExampleCode.HelloWorld"));

      Assert.IsTrue(Gateway.IsValidForMask(
              "ExampleCode.*", "ExampleCode.HelloWorld"));
  }

public class Gateway : IRunner
  {
      public string Mask { get; set; }

      public String ExecuteRequest(Dictionary environment)
      {
          string path = environment["path"];
          AssertValidClass(path);
          IRunner instance =
                (IRunner)Activator.CreateInstance(Type.GetType(path));

          return instance.ExecuteRequest(environment);
       }

       private void AssertValidClass(string path)
       {
          if (!IsValidForMask(Mask, path))
          {
              throw new Exception(String.Format(
                    "Invalid Path '{0}' for Mask '{1}'", path, Mask));
          }
       }

       internal static bool IsValidForMask(String mask, String path)
       {
           mask = mask.Replace(".", "\\.").Replace("*", ".*");
           Regex regex = new Regex(mask);

           return regex.IsMatch(path);
       }
 }

public class Gateway : IRunner
  {
      private string mask;
      private Regex regex;

      public string Mask
      {
         get { return mask; }
         set
         {
            mask = value;
            regex = new Regex( mask.Replace( ".", "\\." ).Replace( "*", ".*" ) );
         }
      }

      public String ExecuteRequest(Dictionary environment)
      {
           string path = environment["path"];
           AssertValidClass(path);
           IRunner instance = (IRunner) Activator.CreateInstance(Type.GetType(path));

           return instance.ExecuteRequest(environment);
       }

       private void AssertValidClass(string path)
       {
           if (!regex.IsMatch(path))
           {
               throw new Exception(String.Format("Invalid Path '{0}' for Mask '{1}'", path, Mask));
           }
       }
 }

[TestFixture]
  public class GatewayBehaviorTests
  {
     [Test]
     private void TestGatewayCallSuccessful()
     {
         string result = Run("ExampleCode.*", "ExampleCode.HelloWorld");
         Assert.IsTrue(result.Contains("Hello World"));
      }

      [Test]
      public void TestGatewayBlocksInvalidMasks()
      {
          AssertValidForMask(false, "Example.*", "ExampleCode.HelloWorld");
          AssertValidForMask(false, "ExampleCode.*.Extras.*", "ExampleCode.HelloWorld");
          AssertValidForMask(true, "ExampleCode.*", "ExampleCode.HelloWorld");
      }

      private static String Run(string mask, string path)
      {
          var gateway = new OptimizedGateway {Mask = mask};
          var enviroment = new Dictionary();
          enviroment.Add("path", path);
          string result = gateway.ExecuteRequest(enviroment);
          return result;
      }

      private void AssertValidForMask(
      bool exceptionExpected, string mask, string path)
      {
          Exception found = null;
          try
          {
             Run(mask, path);
          }
          catch (Exception ex)
          {
             found = ex;
          }
          Assert.AreEqual(exceptionExpected, found == null);
       }
}

public int doSomething(int a, int b) {/*...*/})

doSomething(2,3);

assertEquals(8, doSomething(2,3));

http://twitter.com/mkennedy
http://dotnet.ubbuzz.com/

Today those two things came together in a big way. Introducing DevelopMentor’s Twitter presence:

    @DevelopMentor_
http://twitter.com/developmentor_

I encourage you get out there and follow us! Here you will see all the combined Twitter messages of most of the DevelopMentor instructors as well as a couple of messages from DevelopMentor itself. You’ll get the chance to keep on top of the world as viewed by some of the smartest people I’ve had the chance to work with: the DM instructors! You’ll see which instructor posted any given message with an attribution at the end (either “via @marksm” or ^MS depending on the available space).

I’m excited about this not just because I think it is cool and useful (and has to do with Twitter ), but over the last few days I’m the guy who wrote the back-end systems to make this all go. It was a short but fun project. It’s a real testament to .NET that this was mostly written an hour.

After you follow @DevelopMentor_ you might want to interact with some of the instructors directly. Here’s our Twitter accounts in a single place for your reference:

http://twitter.com/bmaso

http://twitter.com/danamiga

http://twitter.com/jason_diamond

http://twitter.com/markblomsma

http://twitter.com/marksm

http://twitter.com/mauricedb

http://twitter.com/mkennedy

http://twitter.com/richardblewett

http://twitter.com/tonysneed

http://twitter.com/wallacekelly

I hope you enjoy the conversations we’re bound to have on Twitter. Come be part of it.

Signed: @mkennedy

I’ve been using Windows 7 as my sole operating system since Beta 1 in January. I’m completely loving it and I was pleased to see how many apps worked seamlessly on it. One that didn’t and I really miss is Gmail Notifier. No matter how I try, I always get this:

It’s insane to me that $130B company can’t provide any more than this outdated tool for this job, but I digress…

I’ve looked and looked for a replacement and they are either no longer online, are crappy applications, and so on. Finally I decided to take matters into my own hands. Introducing a clean, simple, unobtrusive, and free Gmail notification application that works on Windows 7 – Gmail 7:

Gmail 7 simply runs in your tray and plays the Windows new mail sound when mail comes in to your Gmail account.

New Mail:

No New Mail:

That’s pretty unobtrusive right? Your account info is encrypted and stored in your user profile and all network access uses SSL.

Download all 41 KB of Gmail 7 here: Gmail7.zip
Requires .NET 2.0 (built into Vista and Windows 7)

Just extract the folder to its final resting place and run it – select “Launch at login” if you want that.

It even comes with some cool ways to view your email by double-clicking the tray icon.

Also thanks goes to Ryan Cook for use of his Gmail C# tools as the basis for part of my project.

I’m a huge fan of ASP.NET Routing. It gained popularity as the part of ASP.NET MVC which channels requests for a given URL to the right controller action. In a wise move, Microsoft moved the routing infrastructure out of ASP.NET MVC and
into its own assembly with the release of .NET 3.5 SP1.

With ASP.NET Routing you can construct search engine optimized and human friendly URLs such as these:

Here part of the URL (tag or user) selects the page and part of the URL (everything or codinghorror) are effectively query parameters to the page.

This is well documented in the ASP.NET MVC world running on your server – you can’t get anything done without it in MVC. But what about Windows Azure? What if you don’t want ASP.NET MVC? What if you’re a traditional type of person and want all the goodness that comes with what is now called ASP.NET WebForms (aka “normal ASP.NET”)?

In this brief post, I’ll cover how to use ASP.NET routing and ASP.NET WebForms in Azure. The sample project can be downloaded if you want to follow along. Phil Haack has
written a good post on using routing alongside ASP.NET WebForms so I won’t cover too much background information.

How does this change for Azure?

The short answer is that it doesn’t. If you get routing working for IIS 7 in your web app, you can effectively deploy it to Azure. But the steps always felt convoluted to me when reading others’ write-ups on this. So let’s run through converting a Windows Azure Web Role  essentially a “stock” ASP.NET WebForms app) to use routing in Azure.

First you’ll need the Azure SDK and Visual Studio tools:

1. Next, create a new solution in Visual Studio by choosing Cloud Service->Web and Worker Cloud Service.
2. Add a new Global.asax file to your web role project.
3. Add a reference to System.Web.Routing and System.Web.Abstractions in your web role project.
4. Define a custom class that derives from IRouteHandler which will map URL parameters into the HttpContext for use in your pages:
   

   internal class CustomRoute : IRouteHandler
   {
       public CustomRoute(string virtualPath)
       {
           VirtualPath = virtualPath;
       }
   
       public string VirtualPath { get; private set; }
   
       public IHttpHandler GetHttpHandler(RequestContext requestContext)
       {
           foreach ( var aux in requestContext.RouteData.Values )
           {
               HttpContext.Current.Items[aux.Key] = aux.Value;
           }
   
           return BuildManager.CreateInstanceFromVirtualPath(
                      VirtualPath, typeof( Page ) ) as IHttpHandler;
       }
   }

5. Register these routes in the Application_Start method of your Global.asax:

   protected void Application_Start(object sender, EventArgs e)
   {
       RouteTable.Routes.Add( "ShowName",
              new Route(
               "naming/show/{name}",
               new CustomRoute( "~/ShowName.aspx" )
               ) );
   
       RouteTable.Routes.Add( "CreateAccount",
              new Route(
               "account/begin",
               new CustomRoute( "~/Account.aspx" )
               ) );
   
       RouteTable.Routes.Add( "Home",
              new Route(
               "home",
               new CustomRoute( "~/Default.aspx" )
               ) );
   }

   Now if you run your app, you might expect the routing infrastructure to work. Inside the ASP.NET Dev Server (aka cassini) this will likely work. But in the Azure Development Fabric you’ll see this:

   

   The problem is you need to tell IIS 7.5 to get out of the way and let the request get to ASP.NET.

6. We’ll define a class to short-circuit the IIS validation

   class Iis7RoutingHandler : UrlRoutingHandler
   {
       protected override void VerifyAndProcessRequest(
           IHttpHandler httpHandler, HttpContextBase httpContext)
       {
       }
   }

7. Modify the web.config by adding a handler and module to the system.webServersection:

   ...

   /li>

8. Finally, we need to recover the data passed to the page. For example, in the sample project we have:route: /naming/show/{name}
   example: /naming/show/michael-kennedy

   How will our page access the value of name? Recall that our custom route stashes the values in HttpContext.Current.Items. We’ll just pull them back out as follows in our Page_Load method of our ASPX class:

   LabelName.Text = (string)HttpContext.Current.Items["name"]; 

That’s it! You can see our routes working in our WebForms app running in Azure (well, technically the screenshot is the dev fabric – but it works in the cloud as well):

Download the source and try it for yourself: AzureRoutingSample.zip (136 KB)

http://www.develop.com/content/newsletters/aprilazure

I’ve republished here for my readers. Enjoy!

Developments: Azure Storage

by Michael Kennedy

[Listen to this article as a podcast: Azure-Storage-Article-Kennedy.mp3]

October 27th 2008, Los Angeles CA – It’s 9 AM and Microsoft is hosting PDC (their most forward looking developer conference). Ray Ozzie and company are introducing Windows Azure: A new platform which is their first foray into the nascent world of large-scale utility computing. This scalable and reliable platform-as-a-service functionality is commonly referred to as ”Cloud Computing” because it runs somewhere out there on the Internet.

Computing platforms that rival the reliability of the utility grids (e.g. electric and gas) which we daily take for granted have long been the stuff of dreams.

A few companies have realized this dream – Google and Amazon come to mind as a couple of the rare exceptions who have accomplished this goal. These companies’ web properties seem to handle unbounded amounts of traffic with zero down time. The data centers, redundancies, software engineering and operations know-how required to make this happen are exceedingly expensive. Some reports have Google spending over $2.4 billion (that’s 2,400 million dollars) on data centers in 2007 alone.

Prior to large-scale cloud computing efforts (circa 2005), most of us could only dream of such scalability and reliability. Today we have at least three highly reputable companies offering some kind of pay as you go cloud computing platform – Microsoft, Amazon, and Google.

Microsoft’s Azure is a new comer to the industry. But for .NET developers, it is not to be ignored. Azure allows you to use your existing skills to build essentially the same .NET applications you are familiar with and “deploy them to the cloud.”

These scalable, reliable, and geographically-replicated applications that run on Azure depend on data of course. Virtually all applications we write will be nothing without their underlying data. But if we simply use the tried and true methods of data storage such as the file system or a (single) database server our data is not all that scalable or reliable. Because we cannot have a scalable and reliable application without data, we need a new mechanism for storing and accessing data from our Azure applications.

Enter Azure Storage

Azure storage is the storage component of the Azure platform. It is actually three data services in one:

• Blob Storage – stores unstructured data essentially as a file, limited to 50 GB of data per blob.
• Table Storage – stores structured data that is somewhat like a database. For full database capabilities there is a high level feature called Sql Data Services (SDS).
• Queues – provides interprocess communication functionality between various web and worker roles in your hosted services or even applications running outside of Azure. Queues can pass small xml or binary messages – less than 64 kb per message.

In this article, we will cover just the basics of the three storage services of Windows Azure. I want to give you a sense for what it’s like to program against Azure Storage. At the base level all access to Azure Storage uses pure REST APIs. This means that you can access it from any HTTP enabled platform / language. For example, to download the blob data called “config.xml” in the container called “settings” for the Azure project “kennedy”
you would simply issue a GET to the Uri:

http://kennedy.blob.core.windows.net/settings/config.xml

To save data in a blob you do HTTP POSTs and PUTs in a similar fashion. However, real life is full of edge cases, error handling, security, and serialization which makes the pure HTTP model error prone. Thus, a sample library serves as the de facto .NET API to Azure Storage and ships with the Azure SDK. It is called StorageClient and can be found in default installs here:

C:\Program Files\Windows Azure SDK\v1.0\samples\StorageClient

We will examine working with each of the storage services from the perspective of the StorageClient library - but keep in mind that ultimately this library is a wrapper around a basic and open RESTful API.

Setting The Stage: The Sample Application

To explore Azure Storage I have written a simple photo sharing distributed application. These set of applications allow users to upload photos to a photo sharing site. These photos must be reviewed and approved by moderators of the site. Once approved, the general public can view and interact with the photos. For a concrete example, you could imagine writing a distributed version of the wallpaper sharing site InterfaceLift and
deploying it on Azure in this fashion.

You can download the sample application and follow along if you want to see the full source code and try it out yourself. Just be sure to start the Development Storage utility that comes with the Azure SDK before running the application.

Our distributed application consists of three parts.

A typical use case might be as follows (see diagram below).

1. We upload a photo submission with our uploader application. The photo is uploaded to Azure blob storage and a message is sent via an Azure Message Queue to all available reviewer applications. Additional information about the submitter is associated with the photo in Azure table storage.
2. The reviewer application watches the message queue for new messages. When one arrives, the photo is added to a list of pending submissions. The reviewer can either reject (delete) the submission or approve it – move it to a permanent blob storage location where it will be publicly viewable.
3. Users visit our website and can view all approved photos. This list will change in real-time because it is driven by the reviewer application. The web application simply pulls all photos from the approved photo container in Azure blob storage.

Saving Data: Creating Azure Blobs

To save data to Azure Blob Storage, you must realize blob storage follows the ACE pattern (Authority, Container, Entity) to describe a blob. Authority is simply your Azure solution name. Containers are analogous to folders. And entities are analogous to files.

The listing below is essentially the code that runs when the uploader application uploads a pending image submission to blob storage.

Listing 1:

Sending Notifications: Azure Queuing

In addition to uploading the image to the pending images container in blob storage, we will send a message to a message queue to notify any active or future reviewers of the new submission.

Listing 2.

Saving (More) Data: Structured Storage and Azure Tables

Finally, for the upload application, we must also save some information about the contributor. In Azure Storage we have two reasonable places to store this information.

First, we could save this information directly in blob storage as meta-data associated with the blob itself. This is straightforward and easy. But there is a big limitation: information in this meta-data is not queryable. Suppose I want get all images associated with a single contributor. There is no way in Azure Blob Storage to say give me all the blobs with this filter on the meta-data. You would have to pull the properties of every blob and do the comparison client-side. That’s tantamount to filling a DataSet with “SELECT * FROM PendingImages” and it’s a bad idea.

Instead we will use the third type of Azure Storage: Azure Table Storage. Table Storage allows us to store data with up to 256 properties and query this data as if it were a database. It is exactly what we need for the contributor information. However, you must realize this is not a database. A better mental picture is a durable collection of Dictionary object (as in Dictionary from System.Collections.Generics) with querying built on top. I say this because there is no schema or relational constructs in Azure Table Storage. If you need that, then you’ll want Sql Data Services – a service on top of the core Azure platform.

The code to add an entry to Azure Table Storage does not fit into a single method as it’s driven through the interaction of several classes we must define. Azure Table Storage can be accessed via ADO.NET Data Services (client-side) and this is the method we will use.

First we’ll define a client-side schema for our entry by creating a class called Contributor which derives from the class TableStorageEntity (from the StorageClient library).

Listing 3.

Additionally we must define the tables and queries available to ADO.NET Data Services by created a class derived from TableStorageDataServiceContext and we do that below. We simply have one table called Contributors.

Listing 4.

With those two items in place, we can insert a “row” into Azure Table Storage as follows:

Listing 5.

As for querying Azure Table Storage that is very straight-forward. Because we are using ADO.NET Data Services, querying can be done via LINQ as in “from c in svc.Contributors select c.Name”. Ultimately ADO.NET Data Services is also built on a RESTful API so this translates to the underlying HTTP REST calls. Alternatively, you can use that REST API directly from .NET or any other platform.

Waiting on Queues: The Reviewer Application’s Code

Next, let’s look at how we monitor and pull messages from Azure Queuing. Ultimately we must poll the queue using a RESTful HTTP request. But the StorageClient resurfaces this to us as simple events.

Listing 6.

We won’t cover how we move a blob from the pendingImages blob container to the approvedImages blob container which happens when a reviewer approves an image.
You can look at the sample to see how that is done.

Ultimately It’s About the Website

Finally, let’s look at the web application that actually displays the approved images. We don’t do anything fancy such as paging or error handling that you’d see in a real application. But this will give you a good idea how to work with the blob data as a collection.

Here we’ll create a BlobStorage object and access the BlobContainer approvedImageContainer as we have been in most of the listings. But then instead of saving or reading blobs, we use the ListBlobs method to simply list all the approve images in that container. In order to show the images on our webpage, we just use the BlobProperties.Uri and directly reference that in our HTML. Our ASP.NET application does not touch the data. Rather the consumers (IE, Firefox, Chrome, etc) of the HTML pull the image data directly from blob storage as they would from any web server.

Listing 7.

Now you have a good idea of the concepts and motivation behind Azure Storage. You have seen some typical usages of each of the three storage features: blob storage, table storage, and queuing. Our samples made use of the sample storage API library called StorageClient. Underlying this library we saw that Azure Storage is entirely accessed via RESTful APIs.

Want to get started? Visit http://www.azure.com and choose “Try It Now” to register for a CTP Azure account. You’ll need to download the various SDK’s listed on that same page. They will install the Visual Studio projects required for working with Azure as well as the Development Storage and Development Fabric so you can develop and debug your applications before deploying them to the cloud.

If you want some intensive, expert-lead training on Azure and associated .NET 4.0 topics be sure to contact DevelopMentor. Or call 800.699.1932 to find out what classes we have available today.

WCF-REST-Kennedy-Peepleocity.wmv 35 MB (WMV HD)
WCF-REST-Kennedy-Peepleocity.mp4 96 MB (MP4 / iPad)

I cover building WCF services using REST princples, the WebGet and WebInvoke attributes, working with the SyndicationFeed & Rss20FeedFormatter classes, and configuration-free WCF hosting in IIS.

You can also download the source code of the project built in the screencast.

Finally, if you’re willing to do without video you can download just the audio as an MP3.

http://dotnet.ubbuzz.com

was featured on “This Week on Channel 9” last week.
  (see segment 7:45 – 10:25)

Thank you Dan Fernandez for covering our site! If you’re unfamiliar with .NET Dev Buzz, then what are you waiting for? You’d better check it out!

   Debugging The Future: The Video by Jason Whittington

If you want to debug .NET applications right down to their memory footprint, this talk is for you. If you like this type of presentation, be sure to check out the classes we offer at DevelopMentor.

Download the library with sample test project here:
Kennedy.WorkflowTesting.zip (216 KB)

You can also just jump to the code.

First a Little History:

Last September I posted this teaser entitled Unit Testing Coming to a Workflow Near You. My intention was to post this article that you’re reading now shortly thereafter when I got some free time to polish things up. In that previous post, I highlighted what I could determine to be the current state-of-the-art with regard to unit testing workflows, circa September 2008.

Then I heard through some inside sources that this MSDN Magazine article was about to come out:

    Foundations: Unit Testing Workflows and Activities
    by Matt Milner.

So I decided to see what Matt’s article had to offer to the conversation. It’s a good article to be sure. It covers a lot of the things I thought were undiscussed and yet important to unit testing WF (e.g. using WF services as points of dependency injection for mocks and stubs). Thanks Matt! I don’t have to write about that now, but you’ll see it used in the sample with my library.

What I was really waiting to see was would that article make this post redundant? After reading it, I can say that there’s still a long way to go – and this library will get us most of the way there. Now let’s get to the good stuff!

Significant Advances:

That’s a pretty bold statement, significant advances: let’s see if I can back it up. Here’s what’s missing in one way or another from all the previous work on unit testing WF. (Please note that this discussion is in no way intended to belittle the work of anyone quoted above, just to build on their work and advance testing for us all).

Problems with unit testing WF today that are solved by this library:

1. Testing single activities: Testing single activities in isolation is hard.
2. References to the activity: Direct access to the activity under test for asserting on its properties is nearly impossible.
3. Waiting on workflows: Workflows run on background threads which means waiting for the outcome inside the test method is more cumbersome than necessary (ManualWorkflowScheduler is unnecessarily cumbersome as well).
4. Untyped name/value collections as input: Using untyped name/value collections as input and return values is error prone (for testing and general use).
5. Expected exceptions: Testing “failure as success” cases for error handling is essentially broken for WF: Either the exception type is lost, or the call stack is lost, or both.

The Sample:

Let me set the stage first before we see the test code. I have a somewhat realistic workflow which will exchange two stocks and either debit or credit your bank account with the difference. So you might want to sell 5 shares of Google and buy 10 shares of Microsoft and pocket the difference.

Here’s the workflow which involves 4 different activities:

Testing On Single Activities:

The first thing to test is the individual activities (like BuyStock and DebitAccount). Here’s the code to test selling a stock (some details omitted for simplicity, exact code follows later). This method uses my library class WfRunner for executing the test.

[TestClass] public class WorkflowTests : IDisposable {     private WfRunner wfRunner = new WfRunner();     [TestMethod]     public void SellStockComputesCostCorrectlyTest()     {         StockDTO dto = new StockDTO( 7, "GOOG" );         SellStockActivity sellActivity =             wfRunner.RunSingleActivity( dto );         double price = testStockSvc.LookupPrice( dto.Ticker );         int quantity = dto.Quantity;         Assert.AreEqual( quantity * price, sellActivity.Cost );     }     // ... } 

This test method (SellStockComputesCostCorrectlyTest) is remarkable for several reasons:

1. We are taking a single Activity, not a workflow, and executing it.
2. We are passing a strongly typed DTO (data transfer object) rather than name/value pairs in a Dictionary.
3. Most Remarkably: We are getting the actual instance of the activity returned to us so that we can explore its properties. Notice that we assert on sellActivity.Cost directly:Assert.AreEqual( quantity * price, sellActivity.Cost );

   This is not easy to pull off – WF intentionally hides activities and workflows it creates behind workflow instances (proxies basically). This hack might be bad for production systems, but it *rocks* for unit testing.It means you get intellisense rather than programming against strings in name/value collections.

4. We are not waiting for the workflow to complete or using the ManualWorkflowSchedule. WfRunner is doing that for us because the method RunSingleActivity is a blocking call.

Just so you don’t think I’m trying to pull a fast one: Here’s that same listing with all the gory details left in. Notice how we’re using WF Services for DI with our test stubs.

[TestClass] public class WorkflowTests : IDisposable {     //     // Define some objects that will be used across all tests.     //     private IAccountService testAccountSvc;     private IStockService testStockSvc;     private Account testAccont;     private WfRunner wfRunner = new WfRunner();     public WorkflowTests()     {         //         // Initialize common data for all tests.          // This is basically what the host of the wf-runtime would do         // but we're using test doubles / stubs for our services.         //         Dictionary<string, double> stocks = new Dictionary<string, double>();         stocks.Add( "goog", 350 );         stocks.Add( "msft", 25 );         testAccont = new Account( 1774, 5000 );         this.testStockSvc = new TestStockService( stocks );         this.testAccountSvc = new TestAccountLookup( testAccont );         // Install these services for use by our WF activities.         wfRunner.AddService( testStockSvc );         wfRunner.AddService( testAccountSvc );     }     [TestMethod]     public void SellStockComputesCostCorrectlyTest()     {         StockDTO dto = new StockDTO( 7, "GOOG" );         SellStockActivity sellActivity =             wfRunner.RunSingleActivity( dto );         double price = testStockSvc.LookupPrice( dto.Ticker );         int quantity = dto.Quantity;         Assert.AreEqual( quantity * price, sellActivity.Cost );     }     // ... }

Expected Exceptions

That was pretty awesome huh? We solve several of our problems I identified above (testing single activities, references to the activity, waiting on workflows, and untyped name/value collections as input). The last one to cover is exceptions as success.

When we try to buy a stock and we don’t have enough money, the workflow will throw an InsufficientFundsException. This type is a custom exception created as part of my wf application – it’s not part of .NET. We want to test for this exception:

[TestMethod] [ExpectedException(typeof (InsufficientFundsException))] public void CannotBuyWithInsufficentFundsTest() {     ExchangeStocksDTO dto =         new ExchangeStocksDTO             {                 AccountID = 1774,                 StockToBuy = "MSFT",                 StockToSell = "GOOG",                 SellQuantity = 5,                 BuyQuantity = 7000             };     wfRunner.RunWorkflow( dto ); }

Notice that we’re using the ExpectedException attribute. We just call RunWorkflow as a regular method and we get the exception back as a synchronous error. That is fantastic already. But look at the call stack:

SampleLibrary.InsufficientFundsException: Wrapped excpetion message: Insufficient funds for account 1774.
 ---> SampleLibrary.InsufficientFundsException: Insufficient funds for account 1774.
   at SampleLibrary.Account.Withdrawl(Double amount)
   at SampleLibrary.DebitAccount.Execute(ActivityExecutionContext ctx)
   at System.Workflow.ComponentModel.ActivityExecutor`1.Execute(T activity, ActivityExecutionContext executionContext)
   at System.Workflow.ComponentModel.ActivityExecutor`1.Execute(Activity activity, ActivityExecutionContext executionContext)
   at System.Workflow.ComponentModel.ActivityExecutorOperation.Run(IWorkflowCoreRuntime workflowCoreRuntime)
   at System.Workflow.Runtime.Scheduler.Run()
   --- End of inner exception stack trace ---
   at Kennedy.WorkflowTesting.WfRunner.TransformAndThrowIfRequired(Exception realException)
   at Kennedy.WorkflowTesting.WfRunner.RunWorkflow[T](Dictionary`2 namedArgumentValues)
   at Kennedy.WorkflowTesting.WfRunner.RunWorkflow[T](Object workflowDTO)
   at SampleLibrary.Tests.WorkflowTests.CannotBuyWithInsufficentFundsTest()

The WfRunner class has determined there was an exception. Rather than just rethrowing it and losing the callstack (notice it’s still intact), we do this operation called TransformAndThrowIfRequired. TransformAndThrowIfRequired takes the real exception, uses reflection to recreate another exception of that type and wraps the real exception as the inner exception.

This both preserves the exception type (critical for the ExpectedException behavior)
and the callstack (critical for debugging).

Running these tests inside Visual Studio we get all green!

I hope you find this library adds significant value to unit testing of your Windows Workflows. Personally, I think it makes unit testing of your Windows Workflows practical in the real world.

Enjoy!
Michael

Note: This only applies to WF 3.0/3.5. WF 4.0 which is part of .NET 4 which is shipping the end of 2009 may change this considerably.

To highlight the transition from

unit testing -> TDD -> BDD -> Approval Testing

Dan wrote a parody of the song Let It Be.

The Music Video: Let it BBD

I’m pleased to announce that MSDN Magazine just published my Windows Workflow article entitled

“ASP.NET WORKFLOW: Web Apps That Support Long-Running Operations”

I hope you find it useful and interesting.

.NET Dev Buzz
Bringing you absolutely fresh conversations about .NET and the Microsoft developer community.

http://dotnet.ubbuzz.com/

I’ll follow this up with some highlights of a bunch of interesting .NET concepts being used here.

For starters, consider this. This is a website built in ASP.NET web forms, but there is no view state, no form posts, and no (visible) aspx files. It’s fully RESTful.

Surprisingly, you can deploy your unmanaged apps with ClickOnce. You just need a tiny .NET app to get it started.

Here’s how it works:

1. Take your existing C++ project.
2. Add to the solution a .NET console application.
3. Change the project settings on the console application to “Windows Application”.
4. Write the following code for your Main method of your .NET launcher application:

   static void Main(string[] args) {  try { Process.Start( "TheRealApp.exe" ); } catch ( Exception x ) {  string msg = "Error launch application:\n\n" + x;  string cap = "Error Launching Application"; MessageBox.Show( msg, cap, MessageBoxButtons.OK, MessageBoxIcon.Error ); } } 

5. Set the build path of your C++ app to be in the main folder for your .NET application.
6. Add the C++ app and its libraries (if any) as existing items in the .NET app.
7. Change the build action to “Always Copy” as shown here:
   
8. Then you publish your .NET application and when it runs, TADA!, the C++ app is deployed, versioned, and kept up to date as well.

If you want to try it yourself, you can run this sample application here:

Run Michael’s Useless C++ App via ClickOnce…

You can also download the source.

For more background information, look at these posts.

1. .NET 3.5 Brings Breaking Changes to ThreadPool
2. Breaking Changes in the ThreadPool: The Movie
3. Follow up on .NET 2.0 SP1 ThreadPool Bug
4. More on the ThreadPool Bug in .NET 2.0 SP1

Now go forth and create some threads!

When I talk about LINQ people often ask whether it’s possible to have dynamic queries with LINQ. An example of this is presenting the user with a UI that allows them to optionally filter by some criteria or other and sort by some criteria. This was straightforward if you built-up a SQL string in code.

The compiled, static nature of LINQ makes dynamic queries appear difficult at first. They are not and this video and sample application shows you how to accomplish just this.

 Screen shot of sample application

You can download the source code here:

Kennedy-Dynamic-Linq.zip (88.34 KB)

And you can download the screencast video here:

Kennedy-Dynamic-Linq.wmv (35.7 MB)