Cassandra, Hive, and Hadoop: How We Picked Our Analytics Stack

MarkedUp Logo

This is an archive of a blog post I wrote for the MarkedUp Analytics blog on February 19th, 2013. It’s been a popular post and I’m posting it here in order to preserve it.

When we first made MarkedUp Analytics available on an invite-only basis to back in September we had no idea how quickly the service would be adopted. By the time we completely opened MarkedUp to the public in December, our business was going gangbusters.

But we ran into a massive problem by the end of November: it was clear that RavenDB, our chosen database while we were prototyping our service, wasn’t going to be able to keep growing with us.

So we had to find an alternative database and data analysis system, quickly!

The Nature of Analytic Data

The first place we started was by thinking...

Read More

Scalability Lessons we can Learn from Voat

Voat - have your say

Last weekend I found on the /r/dotnet subreddit, one of the places I frequent for news and happenings related to all things C# / F#.

Voat exists simulataneously as two different things:

  1. The Voat software - an open source, ASP.NET MVC + MSSQL implementation of Reddit AND
  2. - an instance of the software that is owned and operated by its creators.

I’m a huge fan of Reddit - had an account there for years, and seeing such a kick-ass implementation of it written in C# with SignalR, ASP.NET MVC, Web API, and lots of other cool toys gets me really excited!

Voat could be a huge opportunity to expose lots of curious onlookers...

Read More

Talking about Akka.NET and the Actor Model on Hanselminutes and .NET Rocks

I’ve done a bit of a “press tour” for Akka.NET since we released Akka.NET v1.0 at the beginning of April and I wanted to share a couple of the interviews I’ve done in the .NET community since.

Hanselminutes: Inside the Akka.NET open source project and the Actor Model with Aaron Stannard

I did an interview with Scott Hanselman immediately after .NET Fringe and we covered the goals and scope of the Akka.NET project, but also really took a close look at the Actor model and why it’s becoming a popular concept among .NET developers.

Listen to me on Hanselminutes! (33:04).

.NET Rocks: Akka.NET V1 with Aaron Stannard

Carl and Rich, the two hosts of .NET Rocks, did a better job with the synopsis than I could!

Akka.NET ships! Carl and Richard talk with Aaron Stannard about Akka.NET,...

Read More

How to Start Contributing to Open Source Software

The Petabridge team (all two of us) just wrapped up a big two weeks. We launched Akka.NET V1.0 and then traveled to Portland to talk about .NET open source software at .NET Fringe.

.NET Fringe Conference

One of central themes of .NET Fringe is open source communities - and there’s two sides to this coin:

  • How does an open source project successfully attract contributors and make them effective?
  • How does your typical software developer become an OSS contributor?

I’m going to touch on the former topic in a subsequent post, as that appeals to a more niche audience than the latter.

Why do open source?

All OSS contributors start...

Read More

On a Mission to Mars

This weekend I went through the Landmark Forum and had to confront a big secret that I’ve been keeping hidden for virtually my entire life: the true aim of my ambitions.

Ever since I was a middle schooler, I’ve wanted to help build the first human colony on Mars. I’ve multiple hours per day thinking about it for years and years and told no one. I was afraid that people would think it’s a ridiculous goal, and that I wasn’t successful enough to do it yet. And really, I was scared that I would fail. So I let these fears stop me from ever trying or ever sharing it.

The Forum helped me realize that these fears are bullshit.

I’m going to help put the first permanent human settlement on Mars. That’s my mission, and today I’m committing to realizing the possibilities that go with it. Sure, it’ll be...

Read More

Creating a Custom ETW EventSource for Debugging High-Performance Code in C#

One of the things I’ve been working on for both Helios and Akka.NET is a custom ThreadPool implementation that allows us to isolate mission-critical code from the noise of the CLR ThreadPool.

My second pass at implementing one of these instanceable thread pools had me borrow the work-stealing queue implementation that’s been responsible for dramatic performance improvements in the CLR ThreadPool. The original ThreadPool code is very optimized and relies on native calls internally, so it’s not always easy to follow.

My initial performance numbers for DedicatedThreadPool were absolutely awful, so I needed to find a high-performance way of measuring just how well this work queueing structure was working for me.

Enter Event Tracing for Windows (ETW.) I spent way more time than I care to admit trying to figure out how to...

Read More

Want me to work on your distributed .NET applications?

Petabridge - We help .NET companies make distributed, realtime applications.

Time for a brief commercial interruption…

Recently Petabridge announced its professional services offerings around Akka.NET.

I’ve been approached over the past year by a lot of .NET developers who’ve needed help using Helios for socket programming, or Akka.NET for distributed systems programming in .NET, and sometimes just some help getting Cassandra set up on Windows Azure.

Well, as part of Petabridge’s offerings my time is officially now on the market for your projects.

If you would like my personal help learning how to use Akka.Cluster to build scalable ASP.NET and Windows Service microservices , deploying Akka.NET on Azure, rewriting your in-house socket servers to use Helios, or whatever else you need - request...

Read More

The Right Way to do Equality in C#

One of the pitfalls of doing development in C#, Java, C++, or really any predominantly Object Oriented language (OOP) is how “equality” is defined.

In C#, for instance, you have the following methods that are built into every object:

  • object.Equals
  • the == operator
  • ReferenceEquals, for explicitly checking reference equality

My personal opinion: in any managed language, checking for referential equality is a pretty bad default - and if you’re working with immutable objects then equality by reference doesn’t really work.

In C/C++, where pointer arithmetic and knowing the precise location of something in memory matters it’s a different story. Equality by reference is the correct default in that case.

What’s the right thing to do?

Read More

Akka.NET - One Year Later

Akka.NET - Distributed Actor Model for .NET

I originally posted this at Beyond HTTP, Petabridge’s official blog. Read the original there.

This month marks the one-year anniversary of Akka.NET and it has been a massive year for Akka.NET. In December of 2013, Roger Alsing and I had never met and we were both working on our own actor framework implementations in .NET.

Before it was Akka.NET…

My startup at the time, MarkedUp Analytics, was just starting its R&D efforts to begin developing its real-time marketing automation product on top of our existing in-app analytics product. We knew we needed the Actor model...

Read More

My Next Thing: Petabridge - the Future of Distributed Software in .NET

Petabridge - Akka.NET Training and Distributed Computing in .NETAfter wrapping up MarkedUp, I took some time off to consider my future. Travel. See old friends. Catch up on rest.

During the entire time I was gone I received a steady stream of questions and inquiries about two open source projects I helped found:

P.S. Get the latest from Aaronontheweb

Have my most recent essays and articles delivered directly to your mailbox.