Aaronontheweb

Hacking .NET and Startups

New Windows Phone 7 App: Lolcats Professional Pro Plus

December 22, 2011 08:30 by Aaronontheweb in Windows Phone 7 // Tags: , , // Comments (0)

localcatsproplus-icon-largeIn addition to my regular work for Microsoft and the open source projects I create and contribute to on Github, I also like to ship the occasional app or website under my StannardLabs masthead.

My absolute favorite app that I shipped early in the summer is Lolcats Professional Pro Plus, a simple app that makes it a breeze to flip through pictures of Lolcats from ICanHasCheezburger.

I’ve since modified the app pretty drastically so that it now includes touch support, a clean minimalist UI, and a selection of 60,000 plus images to page through, with hundreds more being added every day.

Lolcats Professional Pro Plus is currently designed for Windows Phone 7.0, although a Mango version is on the way soon. Here’s what the app currently looks like:

screenshot1screenshot2screenshot3screenshot4

Do me a favor and go download the app from the Windows Phone 7 app store – it’s free and is a great way to kill time when you need it.

If you enjoyed this post, make sure you subscribe to my RSS feed!



Project Scaffolding for Python

December 18, 2011 11:33 by Aaronontheweb in Python // Tags: , , // Comments (0)

 python-logo.pngOver the summer I went through Zed Shaw’s Learn Python the Hard Way in an effort to broaden my horizons and start learning languages beyond C#.

Learn Python the Hard Way is a book designed for people with no programming experience whatsoever, but even as an experienced software developer I got a lot of value out of it. Zed does a great job forcing the reader to develop Python muscle memory early, and he introduces concepts like unit testing in Python and package management from the beginning.

One of the really valuable lessons of Learn Python the Hard Way is in exercise #46, where Zed recommends a great generic folder and file structure for new Python projects, like this one:

/[projectname]/
/[projectname]/setup.py
/[projectname]/bin
/[projectname]/docs
/[projectname]/[projectname]
/[projectname]/[projectname]/__init__.py
/[projectname]/tests
/[projectname]/tests/__init__.py
/[projectname]/tests/[projectname]_tests.py

This is a great starting point for most Python projects, as it allows you to easily build a Python package with setuptools and to unit test it (using Nose in this case.)

However, I really didn’t like having to manually copy and rename folders / files constantly in order to work with this project structure, so I wrote my own Python package called “Scaffold” which allows you to automatically scaffold new projects with this structure directly from the Python command line, like this:

python -m scaffold -p "projectname" [-d {base directory}]

And with that, you’ll automatically generate the Python project structure I showed you previously, with Nose set up as the default unit testing framework.

Installing Scaffold

Scaffold is compatible with Python 2.7 – I haven’t tested it with Python 3.0+, but I’d be surprised if there were compatibility issues with it given Scaffold’s simplicity.

You can install Scaffold directly via Pip:

pip install scaffold

If you have more questions about Scaffold and its usage, please see the scaffold-py readme on Github.

Contributing to Scaffold

You can find the source to scaffold-py on my Github account. Patches are not only welcome, but strongly encouraged!

If you enjoyed this post, make sure you subscribe to my RSS feed!



Intro to Node.JS for .NET Developers

December 14, 2011 04:51 by Aaronontheweb in Azure, Node // Tags: , , , // Comments (7)

Node.js darkMicrosoft announced out-of-the-box support for Node.JS on Windows Azure on Tuesday; we pushed both an official Node.JS SDK for Windows Azure services (table storage, blob storage, and queues) and some tools for creating and managing Node.JS roles on Windows Azure, both of which are available through the official Windows Azure page on Github.

I’ve been playing around with Node for a short while and have the pleasure of working with some great startups that utilize Node heavily in production, so I wanted an opportunity to explain to my fellow .NET developers why they should be excited about Node support on Azure and how they can put it to work for them.

Node at a Glance

If you want to spend 90 minutes and get a full 100-level understanding of Node, check out The Node Beginner Book. If you’re fine with the footnotes, keep reading.

Node is an asynchronous distributed programming platform built on top of Chrome’s V8 JavaScript engine (the same engine used to parse and execute client-side JavaScript inside Chrome.) Node is actually server-side JavaScript, but its syntax and prose are familiar to every web developer to some extent.

Node’s true innovation is its evented + asynchronous I/O model.

var http = require('http');
http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end(myJavascriptObject.getSomeStatusInfo());
}).listen(1337, "127.0.0.1");

The primary method of any Node application runs a single-threaded continuous event loop (the .listen method of the HTTP server), which farms out long-running tasks to worker threads (the anonymous function) which callback the main event loop once they’re finished:

nodejs for dotnet

There are three major advantages to this model:

  • The main event loop uses a single thread to handle multiple concurrent connections, which makes the overhead of Node.JS grow relatively slowly as the number of requests it has to serve increases as there’s no OS thread / process initialization overhead;
  • All long-running tasks (network I/O, data access, etc…) are always executed asynchronously on top of worker threads which return the results via callback to the event loop thread; and
  • JavaScript’s language features (functions as objects, closures, etc…) and Node’s programming model make this type of asynchronous / concurrent programming much easier to utilize – there’s no thread management, no synchronization mechanisms, and no message-passing nonsense. This eliminates a lot of pitfalls that most developers fall into when attempting to develop concurrent applications.

Using Node.JS with .NET

So, you’re a veteran ASP.NET MVC or WCF developer… Why should you be interested in taking a look at Node?

First, Node is fully supported in environments .NET developers are familiar with. Node runs on Windows; Node can be run as a managed module inside of IIS; and Node can now run on Windows Azure. There’s even a NPM package for having Node.JS interop with SQL Server.

Second, Node does a great job tackling vertical issues like real-time notifications, web sockets, and other scenarios that aren’t easily addressed through existing .NET technologies (although SignalR is pretty cool.)

Node isn’t a replacement for everything – frankly, it’s not a great solution for heavy web + database CRUD applications or for serving static content.

Here are some scenarios that I like for Node:

  • Handling multiple concurrent connections (web sockets, handling events from real-time messaging systems [ZeroMQ / Azure Service Bus], anything with socket programming…)
  • High-volume, small response-size systems – think of a real-time logging system where you have to write millions of messages an hour and only send small ACK response objects back to the caller – Node.JS is perfectly suited to handle this;
  • Real-time Pub/Sub systems – if you have an application that is heavy on notifications and needs to do it in real-time, Node is a great choice given its penchant for getting in and out of requests quickly.

Need a pragmatic example?

Take a close look at Socket.IO, arguably the best solution for working with web sockets and long-polling available on any platform.

If you have an application where real-time notifications are important like a messaging application or a social game, then running Socket.IO alongside IIS on your boxes gives you a significantly simplified solution both on the client (the Socket.IO client is backwards compatible all the way to IE5.5) and the server for handling socket connections and routing, and it takes a minimal amount of resource overhead to do it.

I’m going to be doing some more work around Node.JS and Azure in the New Year, so if you have any questions or comments please feel free to hit me up in the comments here.

Want to get started with Node.JS and Windows Azure?

If you want to try out Node.JS on Azure, sign up for the Windows Azure free trial and install the Node.JS Azure SDK via Web Platform Installer.

If you enjoyed this post, make sure you subscribe to my RSS feed!



How to Modify Machine.config on Windows Azure Web Roles

December 1, 2011 19:04 by Aaronontheweb in Azure // Tags: , , // Comments (0)

Earlier this week I spent some time helping a company troubleshoot some performance issues with Windows Azure – their ASP.NET request queue was growing longer than the maximum amount of queued requests supported by default in IIS (5000) during bursts of high activity, so I had to help them find a way to empty the queue faster.

One way to do this is to modify the number of max/min IO and worker threads available to any web role instance, and these values can only be modified through machine.config settings.

As you know, Windows Azure VMs are not persistent so you can’t modify machine.config and save it to file system like how you might if you were hosting IIS on your own local box.

Thus, I crafted a solution using AppCmd and Windows Azure startup tasks.

Here’s some of the code below:

REM Increases the number of available IIS threads for high performance applications REM Uses the recommended values from http://msdn.microsoft.com/en-us/library/ms998549.aspx#scalenetchapt06_topic8 REM Values may be subject to change depending on your needs REM Assumes you're running on two cores (medium instance on Azure) %windir%\system32\inetsrv\appcmd set config /commit:MACHINE -section:processModel -maxWorkerThreads:100 >> log.txt 2>> err.txt %windir%\system32\inetsrv\appcmd set config /commit:MACHINE -section:processModel -minWorkerThreads:50 >> log.txt 2>> err.txt %windir%\system32\inetsrv\appcmd set config /commit:MACHINE -section:processModel -minIoThreads:50 >> log.txt 2>> err.txt %windir%\system32\inetsrv\appcmd set config /commit:MACHINE -section:processModel -maxIoThreads:100 >> log.txt 2>> err.txt

I’ve open-sourced the rest of my IIS machine.config / app pool scripts on Github. Take a look at them and let me know if you have any questions.

If you enjoyed this post, make sure you subscribe to my RSS feed!



How to Migrate Data between On-Premise SQL Server 2008 R2 and SQL Azure Without Getting a Migraine

November 28, 2011 17:33 by Aaronontheweb in Azure, SQL Server // Tags: , , // Comments (2)

I love my job at Microsoft, but there are some times when we simply make it really damn hard for people to do business with us. Migrating data from an on-premise SQL Server to SQL Azure is sadly one of those lapses where, for whatever reason, we’ve left people who aren’t full time SQL Server DBAs totally lost in the wilderness with a set of poorly documented and often dysfunctional tools.

After spending a couple of hours shaving yaks myself tonight trying to move a ~250mb data set (small, but not trivially small) from one SQL Azure database to another, I thought it would be best if I documented what I did to make it work.

Scenario: I needed to download an expensive dataset on one SQL Azure account and migrate it to another on a different SQL Azure subscription, but I needed to make and test some schema changes against the data set locally before I exported it back to its final resting place.

Seems simple enough, right? I just need to:

  1. Download the SQL Azure Database to my local system;
  2. Make a back-up of the dataset locally in case I screw up the other one during my schema changes;
  3. Finish making / testing / integrating schema changes;
  4. Deploy to SQL Azure on new subscription.

Sounds pretty easy to me, as it would any sane developer who’s had some experience working with the Microsoft server stack since 2000.

Rather than tell you the things I spent hours trying that don’t work I’ll explain what does.

Before you do any of this, make sure your SQL Azure Firewall has an exception for whichever development / backup machine you’re using.

Backing Up SQL Azure to On-Premise SQL Server 2008 R2: RedGate’s SQL Azure Backup

SQL Azure Backup from RedGate solved my first problem painlessly – currently SQL Azure Backup is free so grab a copy while you still can.

All you have to do is point it to your target database on SQL Azure and your target on-premise SQL Server database (screenshot taken from RedGate.)

That’s about as complicated as it gets.

Pushing from SQL Server 2008 R2 to SQL Azure: SQL Server Management Studio Data Export (Requires Magic)

One of the SQL Azure migration techniques Microsoft recommends is using the SQL Server Import and Export Wizard to push your data to SQL Azure. Unfortunately they leave one critical part out, which I will show you.

You begin by selecting your database in Management Studio (I’m using the Express edition.)

image

And for the second step you can leave all of the client / connection settings as-is for your on-premise SQL Server (shown below.)

image

And now we come to the part that the MSDN documentation totally left out… If you’re like me, you’ll naturally try to login using the SQL Server Native Client 10.0 datasource for your export target, because hey, makes sense right?

image

As it turns out, you need to use the .NET Framework Data Provider for SqlServer and change the following fields:

  1. Set Security –- Encrypt to true;
  2. Set Security – Password to your SQL Azure login password;
  3. Set Security – User ID to your SQL Azure user id without the @servername at the end;
  4. Set the Source – Data Source property to the servername.database.windows.net – no need to specify any of the TCP or port nonsense here; and finally
  5. Select the Source -- Initial Catalog to be [SQL Azure database you’ve already created on your service but haven’t necessarily set a schema for yet.]

Once you’ve cleared this hurtle it’s pretty much smooth sailing.

If you have any questions about this process or why I didn’t mention some of the available alternatives, go ahead and ask them in the comments and I’ll get back to you.

If you enjoyed this post, make sure you subscribe to my RSS feed!



Rise of the Popped Collar Programmer

November 19, 2011 10:09 by Aaronontheweb in General // Tags: , , // Comments (14)

I am frankly disturbed by a trend that I’ve seen both in-person and all over Hacker News / Reddit through the past year, and I am going to finally give it a name: “popped collar programming.”

Popped collar programming is the hipsterization of software development, and it’s happening in a co-working space, unprofitable venture-backed startup, or coffee shop near you.

A popped collar programmer’s life begins like this:

  1. [New Technology X] is released and offers a new and interesting view of how to perform [programming chore Y];

  2. Reddit / Hacker News announces the release of  [New Technology X] and is greeted with tons of enthusiasm and applause;

  3. Curious developers check [New Technology X] out – it’s an early release and thus they discover [New Technology X] is missing [Critical Features 1….N] and has [Stability Issues 1…N]; most of the early adopters utilize the technology sparingly in production and only where it’s the right solution for the problems they’re trying to solve.

  4. [Group of early adopters A] can’t get enough of [New Technology X] – they create lots of blog content on how to couple it alongside other popular technologies and receive front-page treatment on Reddit / Hacker News.

  5. [New Technology X], despite lacking [Critical Features 1….N]; having [Stability Issues 1…N]; and often not being the best business case match is now used in 100% of production projects by [fanboys of early adopters A].

  6. [Fanboys of early adopters A] declare the death of [Established Competitive Technology with Massive Following X]; [fanboys of early adopters A] have now become popped-collar programmers.

  7. [Popped Collar Programmers] begin purchase of ironic  / retro t-shirts; growing porn star mustaches;  writing blog entries about the challenges of scaling [New Technology X] despite having a trivial number of users on their service; blog entries make front page of Hacker News.

Popped Collar Programming is essentially “adopting technologies for the sake of appearances” – technologists who’ve fallen victim to this way of thinking often make technology decisions without any regard for what’s the best tool for the job. 

Popped Collars Bro

Once you’ve gone down the road of letting anonymous people on the internet convince you that having cool war stories with cutting edge technologies you can tell on Hacker News is more important than shipping software that works and is easy-to-maintain, you’ve lost the objectivity and pragmatism needed to be an effective software developer.


And it doesn’t stop at technology decisions; popped collar programmers spend more time picking out ironic t-shirts to go with their fedoras and oversize rubber watches than they do logging bugs.

Don’t Be That Guy

Popped Collar Programmers are inefficient (fitting square pegs in round holes from the start) and ineffective (building complex bug-prone architectures themselves to address problems that are already solved with widely-used technologies.)

If that argument’s not enough to convince you to stop yourself or your friends from being popped collar programmers, let’s try a live-by-peer-pressure, die-by-peer-pressure approach: you look like a complete pair of clownshoes in front of people who actually know what they’re doing.

If this sounds like you or someone you know, don’t be that guy. The solution here is to read The Pragmatic Programmer until you can’t stand to look at that Hadoop + Riak + Redis + Clojure-powered blog platform you created without vomiting in disgust.
Don’t. Be. That. Guy.

If you enjoyed this post, make sure you subscribe to my RSS feed!



How to Use Dependency Injection (Ninject) with WCF Services

August 16, 2011 15:32 by Aaronontheweb in .NET, WCF // Tags: , // Comments (1)

I spent a lot of time late last week trying to figure out exactly how to set up dependency injection for a WCF service I was developing on Windows Azure; there’s some documentation out there on how to do it, but it’s not nearly as helpful as I would like it. Thus I decided to document how I made WCF and Ninject play nice with each other, and I even provide a sample template you can use yourself.

Step 1 – Install Ninject.Extensions.WCF via NuGet

The developers who maintain Ninject were kind enough to put together a set of extensions for working with WCF applications specifically, and we can install this via NuGet (install it from CodePlex.)

Simply type this into the “package manager console” in Visual Studio: install-package Ninject.Extensions.Wcf

image

This will automatically grab the appropriate version of Ninject and the other binaries you’ll need to get the WCF ServiceHost to play nice with your service.

Step 2 – Write Your Own Ninject Module

The next step is to set up your dependency injections by writing your own NinjectModule like this:

using Ninject.Modules;

namespace wcf_ninject.DI
{
    public class WCFNinjectModule : NinjectModule
    {
        public override void Load()
        {
            //Injects the constructors of all DI-ed objects 
            //with a LinqToSQL implementation of IRepository
            Bind<IRepository>().To<LinqToSQLRepository>();
        }
    }
}

Step 3 – Add the “Factory” Attribute to Service1.svc

The next thing you need to do is tell the service host to use the Ninject Factory to instantiate the WCF service by way of a custom ServiceHostFactory included in the Ninject WCF Extensions.

To do this, right click on the “Service1.svc” file (or whatever you name your service) and select the “View Markup” option:

image

Once there, simply add this line:

Before

 

<%@ ServiceHost 
    Language="C#" 
    Debug="true" 
    Service="wcf_ninject.Service1" 
    CodeBehind="Service1.svc.cs" %>

After

<%@ ServiceHost 
    Language="C#" 
    Debug="true" 
    Service="wcf_ninject.Service1" 
    CodeBehind="Service1.svc.cs"
    Factory="Ninject.Extensions.Wcf.NinjectServiceHostFactory" %>

Step 4 – Add Global.asax Make It Inherit from NinjectWcfApplication

The final step is to add a Global.asax file to your WCF project and make it inherit from the NinjectWcfApplication class, like this:

public class Global : NinjectWcfApplication

In order for this code to compile, you’ll need to add a “CreateKernel” method to Global.asax – this method is what utilizes your Ninject module and uses it to inject dependencies into your WCF service. Here’s what that code looks like for this example:

protected override IKernel CreateKernel()
{
    return new StandardKernel(new WCFNinjectModule());
}

Now there’s just one last step…

Step 5 – Remove the Default Constructor from Your WCF Service; Have Dependency-Accepting Constructor Instead

Your final constructor in this example should look something like this:

 

 public Service1(IRepository repository)
{
    _repo = repository;
}

Get rid of any default constructors (ones that don’t accept arguments) – Ninject will raise an error if it finds one in the signature of your class as of writing this.

Grab the WCF + Ninject Visual Studio 2010 Project off of Github

You can download the Visual Studio 2010 project that contains this WCF + Ninject sample off of Github if you want to use it as a starting point.

If you enjoyed this post, make sure you subscribe to my RSS feed!



New Open Source Project: MVC.Utilities

August 14, 2011 07:33 by Aaronontheweb in ASP.NET, Open Source // Tags: // Comments (0)

I announced this on Twitter late last week, but I open-sourced a number of common helpers and service interfaces that I use throughout all of my production ASP.NET MVC applications and wrapped it into a project I call MVC.Utilities.

MVC.Utilities has a number of helper classes for the following:

  1. Encryption;
  2. Caching;
  3. Authentication;
  4. Routing;
  5. Security for user-uploaded files;
  6. and display helpers for things like Hacker News-style datetimes.

 

If you want more details and examples of what the library does exactly, be sure to check out the MVC.Utilities Wiki on Github. All of these services are meant to be Dependency-Injected into ASP.NET MVC controllers and are designed as such.

The library has a lot of room to expand and grow, and I encourage your ideas and contributions to the project! Just create a fork of MVC.Utilities on Github and send me a pull request; props to Wesley Tansey for already submitting a couple of patches!

Installing MVC.Utilities via NuGet

MVC.Utilities is available via NuGet – type in the following command via the Package Manager Console in Visual Studio to have it added to your project today:

Install-Package mvc-utilities

If you enjoyed this post, make sure you subscribe to my RSS feed!



How to Recruit a Technical Co-Founder for Your Startup

August 3, 2011 12:54 by Aaronontheweb in Startup // Tags: , // Comments (2)

The LA startup scene is fascinating, having lived and worked in it for a year now - it's a scene teeming with brillaint people with big ideas, and it's starting to attract some major capital from the Bay Area. It has one major issue: a big shortage of technical co-founders.

As a result, people like me get approached fairly regularly by companies that are either trying to recruit me or recruit through me - the vast majority of the time it's a pair of non-technical co-founders looking to third founder aboard, a technical cofounder, to build the MVP prior to raising some money. In my personal experience, the majority of pitches I've received have been poorly calculated and in need of much improvement.

Speaking as a former and future technical founder, I wanted to share my perspective on what non-technical (and technical, for that matter) founders could do differently to try to bring an early tech guy / gal onboard.

Don't Pitch Airtight Ideas; Start a Conversation

The last pitch most technical people entrepreneurial enough to leave a well-compensated job for a startup with no income want hear is one where a product-oriented founder presents an airtight idea that has no room for discussion whatsoever; every single pixel and user interaction has been planned, as has the business model and everything else. They just need a code monkey to build it.

If this sounds like you, start over or hire an outsourcing firm. Founders want to leave their mark on the the business itself and that means all aspects of it, whether it's the branding, the commercial model, or the product's implementation. Sure, a technical co-founder will defer to a commerce person on the specifics of the customer acquisition strategy or the product person on the details of the UX, but that doesn't mean that they want to sit out of those conversations in their entirety.

Any technical co-founder who's willing to take the risk to leave a high-paying job where they do someone else's bidding isn't going to take a not-yet-paying job to do someone else's bidding. They want intellectual co-ownership as much as any of the other founders.

A better way to do this is to start a conversation about a business idea in general, get the people you want as technical co-founders to buy-in and contribute their own ideas into the business, and let them take some degree of intellectual ownership over the project. Don't pitch! Ask questions - get the people you want to co-found with you involved. The amount of work someone like me will do without cashmoney for a project that we "co-own" is dramatically higher than if we were working on "someone else's project" under the same conditions. Read Leadership is an Art and remember what Max Depree says about letting your creative giants roam free; that applies here!

If you're not comfortable letting a geek (or anyone other than yourself) into the idea ownership fold, you're probably not mature enough to found a company.

Build a Demo Yourself

You know what impresses the hell out of me? When a non-technical person has the drive to build an early prototype or a set of wireframes themselves to help illustrate the concept and what they ultimately want to do.

There's still room in the business model and product idea for me to get some mindshare (see point #1,) but the fact that the person was passionate and humble enough to build a cludgy demo that doesn't work quite right tells me that they're not going to flake out and move onto something else while I'm left holding the bag.

A demo is also a great way to get a tech co-founder to buy in, see where improvements can be made, and help them understand the business.

Prove Your Worth

I know I can code, and I can prove it with my Github profile or my portfolio - how do I know that you can aquire customers or design a truly usuable prodct? Go out of your way to demonstrate that you've got skills - get some early partners signed up early; build a blog and a following around whatever idea you have; get some user testominals; do some UX testing; and so forth.

Present an Interesting Challenge

You know what intrigues me, as a developer? Solving big problems and helping a business grow. You know what doesn't? Modifying a WordPress blog for a lame-assed content play. Give me (and other technical types) something big we can chew on - we dig that. It's not that we're looking for overly-complicated solutions for simple problems - we're looking for interesting problems.

Here's how you can tell if your problem is interesting or not: is there an off-the-shelf product that does this? If the answer is "yes," you probably don't need a techincal co-founder.

If you need some advice, swing by Coloft and I'd be happy to talk with you. I'm usually there ;)

 

If you enjoyed this post, make sure you subscribe to my RSS feed!



MongoDB vs. SQL Server 2008: A .NET Developer’s Perspective

June 30, 2011 16:01 by Aaronontheweb in ASP.NET, MongoDB, SQL Server // Tags: // Comments (22)

One of the first projects I put together this year was Captain Obvious, a nifty little application that runs off of AppHarbor and ASP.NET MVC3. What made Captain Obvious special for me was that it was my first time using something other than a relational database1 in production – I chose MongoDB because it stands out to me as a lightweight, easy-to-work with store that’s easier to use for most CRUD applications. Since then I’ve gone on to build other projects which depend on Mongo.

What I’ve learned since is that MongoDB and SQL Server are tools that aren’t 100% interchangeable and are more situational than dogmatists make them out to be.

My goal in writing this is to help inform you on how you should decide to judge these two technologies as options for your ASP.NET / WebMatrix / WCF applications.

Relational Data Models vs. Document Models

The key to using Mongo or SQL Server effectively is understanding how the underlying data model works and how this impacts your ability to read / write what you want when you want to the datastore. Below are illustrations of the relational (SQL) model versus the document model.

mongo vs sql differences

 

In a relational model all of your information is expressed in the form of tables, all of which contain keys and some of which contain foreign keys to other tables. All of the information you read from and write to the database is expressed as either adding rows to these tables or combining their values based on some keys.

In a document model you have a relatively flat collection of items all identified by one primary key2, and instead of defining a relationship between two collections you simply embed one inside the other. So the relationship between the three tables in our relational model is expressed as a single, flat document in this model.

It is important to note here that there’s no schema that tells MongoDB what fields should or shouldn’t be expected in each document in the “Things” collection – in the relational universe each table is strongly, declaratively typed and every single item inserted into the row must conform to all of the constraints imposed by the relational database management system (RDBMS.) Anything goes in Mongo (although this can cause major problems, as we will see later.)

What Do Relational and Document Models Have in Common?

So what do these two data models have in common?

  1. Both support the notion of primary keys and indexes, and MongoDB can support multiple indices if needed;
  2. Both support queries and have models for sorting / limiting results;
  3. Both support the ability to reference other documents / tables (“wha? in Mongo? Yup.”)
  4. Both have a strong typing system; and
  5. Both support aggregation operations like SUM(), COUNT(), etc…

 

Seems pretty straightforward, right? But what’s up with documents being able to refer to each other?

As it turns out, implementing a database where every possible piece of information of interest to users is all embedded inside of its own distinct document comes with some drawbacks, so the creators of MongoDb added support for DbReference and the ability to do cross-collection references as well. A necessary evil in the name of practicality.

What’s Different between Relational and Document Models?

So what’s really different between the two models?

  1. Document models don’t have SQL – their query tools are extremely primitive in contrast (but the models are also much simpler and don't require sophisticated queries;)
  2. Fetching document references in the document model has to be done inside of separate queries3 whereas they can be done all in the same transaction in the relational model;
  3. Document models don’t have a schema – each document in a collection can have extra fields or fields with different type values, whereas all rows must conform to the same set of constraints in order to be inserted;
  4. In the document model types are associated with data upon assignment, rather than declared in advance;
  5. Document models have much more primitive tools for performing aggregations (edit: actually, not necessarily true for Mongo - it has built-in MapReduce which is powerful and sophisticated;) and
  6. Queries are defined on a per-collection basis in the document model, whereas a query in the relational model is an abstraction that simply refers to any number of related tables.

 

The picture that should be starting to form in your head of MongoDb vs. SQL Server at this point is a flat, simple system on one side and a multi-dimensional, rich system on the other. This is how I look at the two technologies in contrast to each other.

What’s Different about Developing .NET Apps Against Mongo and SQL Server 2008?

So at the end of the day, what's the bottom line for .NET developers who want to use Mongo or SQL Server in a web application? What are the REAL trade-offs?

Mongo IS the OR/M

The core strength of Mongo is in its document-view of data, and naturally this can be extended to a "POCO" view of data. Mongo clients like the NoRM Project in .NET will seem astonishingly similar to experienced Fluent NHibernate users, and this is no accident - your POCO data models are simply serialized to BSON and saved in Mongo 1:1. No mappings required.

I'm going to show you two similar pieces of source code I am using in production - one using NoRM and MongoDb on CaptainObvious and the other using Dapper and SQL Azure on XAPFest:

NoRM:

public IdeaResultSet GetIdeasByAuthor(int authorID, int offset = 0, int count = 10)
        {
            using (var db = Mongo.Create(ConnectionString))
            {
                var ideas =
                    db.GetCollection().AsQueryable().Where(x => x.AuthorReference.Id == authorID)
                    .Skip(offset)
                    .OrderByDescending(
                        x => x.DatePosted).Take(count).ToList();

                var totalIdeas = db.GetCollection().AsQueryable().Where(x => x.AuthorReference.Id == authorID).Count();

                //Fetch the referenced authors before we serve up the list again
                foreach (var idea in ideas)
                {
                    idea.Author = idea.AuthorReference.Fetch(() => db);
                }

                var resultSet = new IdeaResultSet { Ideas = ideas, 
                    PaginationValues = new PaginationTuple { 
                        MaxPages = PageCounterHelper.GetPageCount(totalIdeas, count), 
                        CurrentPage = PageCounterHelper.GetPageCount(offset, count) } 
                };

                return resultSet;
            }
        }

Dapper:

public IList<XfstApp> GetAppsByUser(string userName)
        {
            using (var conn = new SqlConnection(ConnectionString))
            {
                try
                {
                    conn.Open();

                    var appResults = conn
                        .Query(@"SELECT * FROM Apps
                                            INNER JOIN AppOwners ON AppOwners.AppName = Apps.AppName
                                            WHERE LOWER(AppOwners.UserName) = LOWER(@UserName)
                                            ORDER BY Apps.DateCreated ASC", new { UserName = userName });

                    //Return whatever we were able to collect
                    return appResults.Select(x => x.ToApp()).ToList();
                }
                catch (SqlException ex)
                {
                    TraceError(ex);
                    return new List<XfstApp>();
                }
                finally
                {
                    //Close the connection when we're finished, regardless of what happened
                    conn.Close();
                }
            }
        }

 

The amount of source code for these two technologies or the nature of it isn't wholly different.... What is drastically different is how I was thinking about the data when I was writing this code - I can save an instance of an Idea object to MongoDb on CaptainObvious without ever having created the collection first or defined a schema.

Whenever I want to look up an idea, I just pick one based off of a key value that I specify and I don't worry about any joins or anything (although I do have to load objects from the author collection if I need to display the author's name and contact info.)

In the SQL universe, I have to define my tables in advance and each time I want to extract an object, I have to think of it in terms of combined relationships between my tables and this requires a more thoughtful approach.

Mongo, in other words, lends itself to rapid application development whereas SQL Server has some innate friction built into any schema-based system.

In Mongo, the DBMS Isn't There to Protect Your Data's Integrity

One of the major advantages of a schema-based DBMS is that if the data a calling application tries to insert something that doesn't fit the schema into a row, the operation always fails. In Mongo, this isn't true - you can have one record in a collection with extra fields or fields of an odd type, and it can totally screw up the BSON serializer when it tries to process the collection (depending upon how flexible the serializer is.)

SQL users take this for granted, but when you have issues in Mongo along these lines they can be really frustrating to solve and difficult to debug.

In Mongo, Operations May Not Be Atomic

Operations are not atomic in Mongo by default, so all sorts of fun things can happen when you have multiple users changing properties on the same document. You can set an atomic flag to true, but even then operations still aren't really atomic (they're written from memory to disc in bulk.)

If you use Mongo and carry the same ACID assumptions that we learned on SQL Server, you might be in for a nasty surprise :p

Conclusion

Overall, the biggest difference between these two technologies is the model and how developers have to think about their data. Mongo is better suited to rapid application development, but in my opinion falls apart in scenarios where ACID-compliant systems are a must, like anything that goes anywhere near a financial transaction.

But, that's just my opinion :p


1typically I’ve only used SQL Server / MySQL in the past

2You can add indices to other fields in Mongo too, but that’s outside the scope of this article

3unless I am doing it wrong, which I may well be doing

If you enjoyed this post, make sure you subscribe to my RSS feed!



Search

About

My name is Aaron, I'm an entrepreneur and a .NET developer who develops web, cloud, and mobile applications.

I left Microsoft recently to start my own company, MarkedUp - we provide analytics for desktop developers, focusing initially on Windows 8 developers.

You can find me on Twitter or on Github!

Recent Comments

Comment RSS

Sign in