Aaronontheweb

Hacking .NET and Startups

Cassandra Summit Talk: Real Time Analytics with Cassandra, Hive, and Solr

July 18, 2013 04:41 by Aaronontheweb in Cassandra, Hadoop // Tags: , , , // Comments (0)

I spoke at the Cassandra Summit this year about how we use Cassandra, Hive, and Solr in production at MarkedUp Analytics.

Planet Cassandra recently made all of the videos and slides available and I thought I would share. Enjoy!

Slides:

If you enjoyed this post, make sure you subscribe to my RSS feed!



Brush the Dust Away

July 15, 2013 04:27 by Aaronontheweb in General // Tags: , // Comments (0)

This post is about stress.

It's been nearly a year since I started MarkedUp Analytics, and we've come a long way - I've raised money, won the business of some amazing customers, built the nucleus of a really talented team, and developed a product that people rave about. At the previous //BUILD conference, no one had ever heard of us - this time our logo was shown onstage during a keynote. Feels like progress; none of it was easy.

I took my first vacation in two years last week and had a chance to put some distance between myself and MarkedUp for the first time since inception, and I thought a lot about some of the assumptions I had about starting a company when I was just getting started, in particular this section from "The Seven Unproductive Habits of Startup Founders:"

7. Sacrificing nearly all social, physical, and personal needs in the name of “getting it done”

You can push yourself to a large volume of work over a short period of time – you can cram 80 hours into a work week if you can try. But you can’t do it forever.

You know what a successful entrepreneur needs?

They need to work out and get sleep so they can maintain their stamina and energy level. They need to go out with friends and blow off steam. They need to go out on dates. They need “me” time to collect themselves.

Creating a balanced life takes discipline – I dropped the ball on this epically at Microsoft. I starting changing my behavior before I left Microsoft and it’s made me more effective and productive.

Counter point: I've put three inches on my waistline since we went live with customers on September 29th, 2012. I stopped working out within a week of our product launch and burned the candle from both ends often since. It's been nearly 300 days since I last synced my Fitbit. My diet consists largely of whatever-I-can-get-in-5-minutes choices, which doesn't correspond well with healthy ones.

Why did I not heed my own advice?

Adding customers as stakeholders to your product produces an immediate and intense set of external pressures on your company. The timelines for delivering everything increase by an order of magnitude and the pressure can be so tremendous that it can crush some employees and founders like empty beer cans. Add investors and a fixed amount of operating capital to the mix and it intensifies the pressure and the timelines by another order of magnitude. Even if you don't panic, you have to move and act like you are.

The immediacy of it all never goes away; it's there waiting for you when you get home; when you sleep; when you're in the shower; when you're out with your friends; when you're in the car; when you're with your family; when you're alone. You are damned by the contents of your inbox; the social media mentions; the Github issues list; the sales inquiries; the status updates from your teammates; the error rate alerts from your monitoring systems; the bid alerts from AdWords; the investor email updates; accounts payable; quarterly taxes; insurance; rent; lawyers; accountants; interviewees; PR; fuck. Your life quickly becomes a list of things you failed to do when you intended to do them.

You can never distance yourself from the pressure; there's no one to cover for you. You are haunted. You are cursed. And if you don't stop running as hard as you can to keep up, your company dies, your hopes and confidence with it.

At least, that's how it's felt for the past 8 months.

Your salvation is in accepting that you can never do all of the things and everything always takes longer than you anticipate. In your role as a highly accountable person you have to give yourself permission to leave things in an incomplete state each and every day. The stress you experience throughout your life is an unconscious choice to self-flagellate and self-punish. It may feel real, but isn't.

You will always harbor doubts; you will always have deadlines; you will always have some negative outcomes; you will have disappointments; and you will always have the ability to choose your priorities and act accordingly.

The inflection point for me was when I spoke onstage at a conference recently and saw just how large I looked in the recording, like one of those retired NFL linebackers who forgot to change his dietary habits after leaving the football behind.

So I wage a campaign against demons of my own making in order to reclaim my time and life. I will not judge myself or my company so harshly. I will not deny myself a glass of red wine with my girlfriend because we have a deadline. And I will get my ass back in the gym, even if it means coming into the office later each morning. I will not beat myself up over unread emails in my inbox, because there's no value in it.

Your accountabilities will always be there for you; what's left of your life and youth will not.

You'll have a list of things to do that is temporally impossible to satisfy; accept it and don't let it slowly strangle the other parts of your life like weeds.

Give yourself permission to go home with a full to-do list each day. Give yourself the permission to enjoy the things that matter to you but don't cross anything off the list. Get used to having people bitch at you for not responding to their emails ("my free time belongs to my kids," as one of my investors put it.) Who cares? Those feelings of guilt and shame are an illusion. Keep your priorities organized, your head clear, and your chin up.

If you enjoyed this post, make sure you subscribe to my RSS feed!



What Do You Need to Become an Elite Developer?

MarkedUp Analytics’s customers are developers and so are most of the people who work at the company, so I spend a lot of time thinking about and refining my answers to the following two questions:

  • What are the lowest common denominator skills and programming knowledge that my customers have?
  • What are the lowest common denominator skills and traits that the developers on my team need to have in order to best serve our customers?
    As I’ve been exposed to more customers and as I’ve turned over members of our team1 I’ve started refining my answers to these questions.
    Turns out that most of our customers are fairly sophisticated and thus I expect the developers on our team to be one step ahead of them.

I’m sure every company says this, but in our case it’s true: we need elite developers in order to solve our customer’s problems.

MarkedUp builds distributed systems that integrate into native third-party applications, written by our customers, on multiple platforms and use machine learning to create insights in as short a timeframe as possible, a task that’s many orders of magnitude more challenging than your average iPhone or web application.

There are companies like DataStax or Liquid Robotics that have to deal with radically harder problems than ours, and I’m amazed by their ability to attract really great engineering talent. So this problem is solvable, but how do I identify elite developers who are going to be able to contribute to the business and tackle these challenges?

I’m still figuring it out, and I’ve made some big mistakes along the way thus far. But I’ve had some successes in attracting great engineering talent to our company too. Here’s my opinion on what makes a software developer elite (your mileage may vary!)

Traits

An elite developer is determined more by their character, attitude, intellect, and diligence more than anything else. Knowing any specific framework, programming language, design pattern, development platform, and so forth isn’t nearly as relevant.

So what character traits does an elite developer have?

Humility – the best developers I’ve worked with know that they are talented but don’t know just how good they are relative to their peers. Even if they did know, it wouldn’t change their attitude. They do the best job that they can and do it knowing that their work isn’t perfect; contains bugs; might not completely meet the spec; and might be changed in the future.

Able to accept criticism – part and parcel with humility is the ability to accept criticism, whether it comes from a peer, a customer, a manager, or anyone else. Whenever you do anything of consequence you should expect criticism from some corner of the universe. Elite developers not only accept it, but they listen to it and try to distill actionable insights if possible.

Doesn’t let good be the enemy of perfect – a lot of talented developers struggle with perfectionism, myself included; over time the great ones learn to live with impurities and get software deployed to production while the perfectionists piss time away on remote edge cases and other activities with no economic value.

Inquisitive – some of our interview questions at MarkedUp are live-coding exercises, and here’s a little secret: we really don’t care very much about the quality of the code produced by a candidate during an interview.

But we care a lot about the questions a candidate asks about the programming challenge.

If a candidate starts coding right away without asking us any questions about the scope of the challenge or the technical details, it’s usually an instant fail. Inquisitive developers do a much better job of learning how their code fits inside a larger system and learning about the real-world use cases for it.

Independent learners – the developers who’ve delivered the most value for me are ones who constantly want to learn and improve their skill set; they ask if they can pair-program with me while I work on our API or on the SDK; independently review some of our code and create JIRA issues for me when they find bugs with my own code (!); they buy O’Reilly books on some of the technologies we use, like Hive and Hadoop, and read them over lunch; and they ask if they can leave work early to attend a tech talk or even give one themselves (!).

Everyone needs help along the way, but the truly great developers are the ones who learn for the sake of learning and the satisfaction of self-improvement. It’s humbling to work with this type of developer, because typically they get other members of the team excited about learning too and turn this into a group activity.

Able to break down large projects into small tasks – people don’t launch rockets or build massive systems by setting out to do everything all at once; they break down massive projects into smaller milestones and small, fine-grained tasks. Really strong developers have the ability to visualize the end in mind and the steps for getting there. It’s an essential skill for an architect of any sort.

Able to visualize relationships between abstractions – part and parcel with the previous point, really great developers have the ability to visualize relationships between abstractions in the systems they work with and build. This can be as simple as being able to see how services interact with each-other in the course of fulfilling an end-user’s request or how classes should be composed inside any given system. This allows a developer to work on a small part of a really large system and understand how their work will integrate with components that aren’t finished or haven’t been started yet.

Embraces challenges – great developers can operate outside their comfort zone, even if it means failing initially. Challenges are viewed as an opportunity to learn and improve, not as a threat to appearance and performance.

Focuses on immediate results – sticking with the previous analogy about launching rockets, great developers break up their work into small deliverables and ship small pieces immediately and frequently into a development branch. They don’t hoard commits or wait until “it’s done.” They ship small deliverables often.

Behaviors

The traits above are the most important ingredients of a great developer, but there are learned and acquired behaviors that every developer must develop before they can become elite.

Unconsciously, religiously commits everything source control – the greatest sin you can commit as a developer is failing to put something into source control for you deploy it. Most companies try to automate deployment in such a way that it’s impossible for deployment to occur without source control involvement, but this isn’t always feasible. Little things like shell scripts, SQL queries, configuration files, and others often fall between the cracks since they aren’t considered part of “the app.”

However, a great developer is religious in their use of source control – anything that gets used in the course of regular business, even a one-off business intelligence script, finds its way into source control when an elite developer is behind the wheel. They rack up pull requests and HipChat spam with religious fervor, because they live in perpetual fear of being the stupid asshole who forced someone else to reverse-engineer a critical backup / restore script during an emergency.

They master their source control system, make liberal use of feature branches and version tags, and clean up after themselves once a feature goes into production. Source control is as natural to them as breathing.

Writes README files, comments, and docs even for internal tools – I tell every developer this when they start at MarkedUp “your code will be a part of this organization longer than you will.” The point of that is that people change careers, companies, and jobs frequently – so when we write a piece of code that does something important, we need to make it possible for someone else to use and modify it without us present to answer questions.

Great developers recognize this and make it easy for other developers to leverage their work, during and after their time on the job.

Embraces bug tracking and agile processes – organizing your work is just as important as performing it, and bug tracking / agile tools like Pivotal Tracker and JIRA enable teams to prioritize and distribute workloads across large projects.

But more importantly, they allow PMs and peers to see what work has been completed / performed without having to hold numerous meetings or calls. Bug tracking, pull requests, and agile boards are communication tools – and great developers embrace them for efficacy in measuring, quantifying, and communicating progress.

Masters debugging, tracing, and monitoring methods – just because the code is in production doesn’t mean it’s finished. Great developers master the tools they use for debugging, logging, tracing, and monitoring apps that are in production or are about to be.

In the .NET universe, this means mastering the Visual Studio debugger, Fiddler, IntelliTrace, the Windows Event Viewer, performance counters, and others. In the OSS world this might mean performance monitoring tools like Graphite and Nagios, or others.

Great developers understand the necessity of these tools and master them, which enables them to respond more quickly to infrastructure failures and bugs.

Communicates concisely and asks relevant questions – most developers are not social butterflies; like any STEM field, it’s often populated with introverted people who like to spend most of their working hours concentrating on issues at-hand.

However, the elite developers are ones who recognize what needs and doesn’t need to be communicated – if there’s something contradictory in a customer spec, it needs to be raised with the stakeholder. If someone broke convention and used camel case instead of PASCAL, just go fix it and send the offender a note, not the entire team alias.

Engages other developers on the team – programming is a team sport, and elite developers love working with other people whom they can help and who can help them. They ask for and solicit help when necessary.

Programming Skills

Now for the programming part – what skills does a programmer need to have before he or she can be considered elite? This is subjective and sometimes domain-specific, but there’s my take.

Experience with multiple languages and platforms – diverse taste in tools is the greatest skill that a developer can have, because it keeps the developer on a position where they know how to learn new things if necessary (and it often is!)

Being a one-trick pony and limiting yourself to a single platform will naturally limit your ability to grow and learn, so great developers try their hands at new things even if it’s just for fun.

Understands language / framework implementations – the most powerful .NET developers I met prior to joining Microsoft all highly recommended that I read this book: CLR via C#. I did, and the content from this book was directly responsible for passing my technical interview at Microsoft. Above that, it’s changed the way I look at all of my .NET code.

Understanding your how code is executed at the OS or hardware level can provide some huge benefits, especially if you’re doing a lot of work with concurrency or native programming. For instance, understanding how SQL Server’s row / table locking algorithms and its journaling system work allow you to design around potential performance bottlenecks in high-performance systems. Great developers learn these implementation details and leverage that knowledge to write more robust code.

Writes code for patterns, not specific instances – it’s an intuitive skill that can’t really be learned, in my opinion. You either have it or you don’t. Really talented developers infer the patterns in a system, even before a module or a piece of code is written and they code for those patterns, rather than specific instances. This skill limits technical debt and results in a tighter, more usable codebase.

Knows how to learn – this might seem obvious, but I’ll reiterate: there are a lot of developers who simply do not know how to learn something new. They can ape examples they’ve seen before and often need to be hand-held along the way, and they can’t truly internalize a new concept and form the muscle memory to code independently with new technologies.

Great developers know their mind and know how to teach it something new – what works for one developer may not work for another, but nonetheless they all know how ot learn.

Has experience with multiple types of software (native, distributed, web, etc…) – a great developer is a well-rounded developer; if your only experience doing software development is with simple CRUD applications, then you’re missing out on a lot of the corpus of modern software development. A great developer isn’t necessarily an adept professional in each domain area of programming – they might focus primarily on web development and dabble in native, but they take away new lessons from each experience and understand more about computing overall.

Until I wrote the MarkedUp SDK, I never had any practical experience with advanced data structures in production like concurrent binary heaps and circular buffers; I likely would never have encountered those had I stuck to doing just ASP.NET MVC or Node.JS development all the time. Now that I have that experience, who knows – I might find some applications for those in a distributed program or a Web API application in the future.

Experience with multiple types of development tools (IDEs, Text Editors, command line compilers, and more) – I have some bad news: if you’re an EMACS ONLY or VS2012 ONLY developer, you’re limiting yourself. Experience with multiple types of development tools is a good thing, because great developers recognize that improving your workflow habits over time is just as important as improving your programming knowledge and experience.

The right tool for the job will often be different – Visual Studio, for instance, is a fantastic tool for when I’m debugging C# or C++ but I much prefer Google Chrome developer tools and Fiddler for debugging JavaScript.

Some sysadmin / operational knowledge – writing the code is only the first step; deploying it or releasing it comes next. Great developers are not often expert system administrators also, but they know enough about their operating environment to avoid pitfalls and execute a basic deployment if necessary.

Knowing how to run a build / release process, deploy a piece of server software to production, or how to configure a piece of COTS software like Apache or IIS to host a web application is critical to ensuring that the software actually meets its functional requirements. If you aren’t able to produce a development build of a native app using compiler flags, then you can’t very well participate in that part of the QA process, can you?

The point is – great developers don’t wait for someone else to solve their problems; they can do enough sysadmin work to build and test on their local environment, and are self-aware enough to not proceed any further if they don’t know enough to run a live deployment.


1developer turnover is a natural and expected byproduct of growth in a technology company, I’ve come to learn. Don’t let anyone tell you otherwise.

If you enjoyed this post, make sure you subscribe to my RSS feed!



Live by “Fuck you, pay me;” Die by “Fuck you, pay me”

March 25, 2013 18:57 by Aaronontheweb in Development Teams, Hiring, Startup // Tags: , // Comments (0)

I came across a blog post by Michael Halligan on Hacker News last week entitled “Benefits matter, or why I won’t work for your YCombinator start-up.” As a fledging entrepreneur trying to attract senior engineering talent to my startup, his post bothered me immensely. I spent about a week mulling it over before I decided to write this.

Michael’s attitude in the post is abrasive, needlessly cynical, and mercenary, but well-argued on its pure economic merits. He describes himself on his own blog thusly:

I'm just this guy who used to enjoy tech, now I would rather build bicycles. I still work in the tech industry, for the money, but I no longer call myself a "technologist".

And he ends his blog post with this call to action for other senior technologists like himself:

To all of my colleagues with whom I have been in the trenches, please repeat after me, "Fuck you, pay me."

This paints a picture of someone who’s been burned before by startup flameouts; who’s put in 100 hour weeks without seeing any life-changing financial benefit; who has real financial obligations like children to put through school; and lastly, a little burned out with his career.

It wouldn’t have bothered me so much were it not for the boorish cheerleading from scores of Hacker News commenters, which indicated to me that this attitude may be more widely held than I would have believed (more on that in a second.)

I really appreciate Michael being so honest about it, because I suspect that this is an attitude that neither he nor any of the supportive Hacker News commenters have ever had the balls to actually express in a verbal salary negotiation with a flesh-and-blood human being like me sitting across the table.

What bothers me about Michael’s post and the Hacker News comments is the cynical view of founders and employers; it dehumanizes them to self-interested profiteers who don’t give a shit about the people who work for them. And the attitude I grok from Michael’s post is “developers: don’t give a shit about [your employer / the founders] either – tell them ‘fuck you, pay me.’”

Let me show you why this bothers me. Here is my honest to God list of things I care about with respect to MarkedUp in order of importance:

1. My personal integrity – because startups are not things of life or death themselves, but integrity is one of the few character traits that outlives the man or woman.

2. The financial health of the company – because a failing company isn’t in a position to serve anyone well, whether they be founders, customers, employees, or investors.

3. The well-being of MarkedUp’s employees – because they are present and future of the business, and I’m going to spend more time with them than my own family.

4. The experience of our customers – because our customers trust us with important information they need to run their own businesses, and we are our own customer.

5. My personal well-being – because I have I dreams of my own but I know that getting items 1-4 right on this list leads to the realization of #5.

I’m sure that if Michael or any of the Hacker News commenters read this list they’d scream “bullshit!”

I treat the growth and well-being of my employees as a non-negotiable requirement for my own happiness and satisfaction, but that conflicts with the Hacker News worldview of the VC-land profiteering dipshit douchebag 27 year old founder (I am, in fact, 27.)

I hire people whom I want to invest in – sure, like any startup we have short-term needs that have to be filled. But the long term health of any business that’s being built to last depends on a high trust culture – where employers and employees interdepend on each other.

I’d crawl over glass for the full time people on my team who take work off of my plate and enable the business to grow – I need them. They support me, so I’d better damn well support them when they need it. I’m making sure they get important benefits like healthcare and the best compensation that I can afford – everything within the bounds of what’s economically feasible for an early stage company of our size and funding level.

I’m invested in MarkedUp’s employees – and when we’re not able to support an employee’s needs I feel like I’ve personally failed as a manager and employer.

So this brings me back to “Fuck you, pay me.” I would never hire anyone who doesn’t want to invest in our product, the other people on the team, and in the company as a whole – because I can’t depend on them to be there for me and everyone else when things aren’t going perfectly. They’d be the first to bounce and the first ones to put the screws on me and their teammates in a bind. I’d never be able to make a person with this attitude happy, so I’d never try.

“Fuck you, pay me” is a self-fulfilling prophecy that will doom you to work for exactly the sort of employers this attitude is supposed to save you from – the kind who won’t invest in you and don’t care about you. When you open the conversation with an employer with “fuck you, pay me” you’re telling them that you don’t give a shit about the product or people, so why should they care about your well being again?

You’ll be passed over for every promotion, every project of any import, any conference or occasional perk, and your opinion will not be solicited or valued on anything that doesn’t pertain strictly to your role. Your teammates will resent you for being unavailable and disinterested. Your managers and executives will treat you with all of the mercenary courtesy and cynicism with which you’ve treated them.

A job is a lot more than a salary and benefits – it’s about enabling people to be happy, employers and employees both. Anyone who comes to work with the attitude of “fuck you, pay me” is miserable by default, an assertion you can validate with a single read over the front page of Michael’s blog. There is little I can do as an employer for someone who is innately miserable. And that makes me sad and frustrated.

Throughout the course of your career you will inevitably deal with shithead employers who don’t appreciate you – quit and move on. Don’t make the situation worse by deciding not to appreciate all employers regardless of who they are – you’ll just alienate the ones who actually want to help you even if they can’t match another employment offer you have on the table.

If money and benefits are literally the only thing that matters to an employee, then they’re worse than the self-interested greedy founder stereotype described earlier. At least the greedy self-interested founders had the balls to take some risks.

So look at your employment opportunities this way – there’s a lot more to working somewhere than just the tangibles. That’s why developers, sales people, and marketers regularly take below market salaries to work at startups – because they get the rare chance to shape the work environment into one that will make them happy.

Happiness is what matters, and benefits + salary shouldn’t be the only factor at the expense of culture, colleagues, creativity, and self-ownership.

If you enjoyed this post, make sure you subscribe to my RSS feed!



Migrating from RavenDB to Cassandra

February 20, 2013 11:30 by Aaronontheweb in RavenDB, Cassandra // Tags: , , // Comments (4)

Today on the MarkedUp Analytics Blog I authored a post entitled “Cassandra, Hive, and Hadoop: How We Picked Our Analytics Stack.”

In it I explain MarkedUp’s evaluation process for choosing a new database, how we selected Cassandra, and some benchmarks from our test. If you want to learn more, go read it.

I’ve written about RavenDB in the past on and I’ve spoken about it at code camp before. I think it’s a great technology for prototyping and it has some really interesting concepts not found in any other database that make Raven really simple and elegant to operate.

So I wanted to copy over a section from our blog post about building MarkedUp on Cassandra which explains why we ultimately needed to move away from RavenDB:

Looking back to what went wrong with RavenDB, we determined that it was fundamentally flawed in the following ways:

  • Raven’s indexing system is very expensive on disk, which makes it difficult to scale vertically – even on SSDs Raven’s indexing system would keep indexes stale by as much as three or four days;
  • Raven’s map/reduce system requires re-aggregation once it’s written by our data collection API, which works great at low volumes but scales at an inverted ratio to data growth – the more people using us, the worse the performance gets for everyone;
  • Raven’s sharding system is really more of a hack at the client level which marries your network topology to your data, which is a really bad design choice – it literally appends the ID of your server to all document identifiers;
  • Raven’s sharding system actually makes read performance on indices orders of magnitude worse (has to hit every server in the cluster on every request to an index) and doesn’t alleviate any issues with writing to indexes – no benefit there;
  • Raven’s map/reduce pipeline was too simplistic, which stopped us from being able to do some more in-depth queries that we wanted; and
  • We had to figure out everything related to RavenDB on our own – we even had to write our own backup software and our own indexing-building tool for RavenDB; there’s very little in the way of a RavenDB ecosystem.

So based on all of this, we decided that our next database system needed to be capable of:

  1. Integrating with Hadoop and the Hadoop ecosystem, so we could get more powerful map/reduce capabilities;
  2. “Linear” hardware scale – make it easy for us to increase our service’s capacity with better / more hardware;
  3. Aggregate-on-write – eliminate the need to constantly iterate over our data set;
  4. Utilizing higher I/O – it’s difficult to get RavenDB to move any of its I/O to memory, hence why it’s so hard on disk;
  5. Fast setup time – need to be able to move quickly;
  6. Great ecosystem support – we don’t want to be the biggest company using whatever database we pick next.

So there you have it. If you want to learn more about our migration to Cassandra, check out our original blog post.

If you enjoyed this post, make sure you subscribe to my RSS feed!



10 Reasons Why You’re Failing to Realize Your Potential as a Developer

Since going full-time on my own startup 6 months ago, I’ve spent a lot of my time recruiting, evaluating, and working with a lot of different developers. My startup, MarkedUp, is an analytics provider for Windows 8 apps (and eventually, other platforms.)

Our technology is sophisticated and difficult even for large companies to master, so the bar is pretty high for developers who join our team. We’re selective about who we work with, but even with careful recruiting practices it’s still difficult to find developers talented enough to pull their own weight on a complex product like ours.

So I want to help average developers who want to improve by calling out 10 performance-killing behaviors that stagnate the careers of most developers.

Most developers stagnate both intellectually and productively after 4-5 years in industry; they adopt some tools, pick up some patterns, learn a language or two, and maybe they’re even able to work at a successful company and contribute to some important products. Great!

But what happens when you hand a developer a blank sheet of paper and the opportunity for them to own a product?

Most of the time: chaos and failure.

Many developers fantasize about being able to own a product and make big technical decisions, but the truth of the matter is most of developers can’t handle anything more complicated than building formulaic CRUD Rails websites using the same 30 gems each and every time.

So what are the behaviors that limit the growth of average developers?

You’re too afraid of failure to learn new tools / languages / frameworks.

During my first quarter at Microsoft I gave a talk to the LA .NET User’s Group on ASP.NET MVC3, which was in beta at the time. For the most part the attendees asked me thoughtful questions and showed a legitimate interest in what the ASP.NET team was cooking up in Redmond.

However, there was a trio of attendees who kept asking me a very suspicious set of questions:

“What are the weaknesses of ASP.NET MVC3?”

“Isn’t ASP.NET MVC too unstable for use in the enterprise?”

“But doesn’t ASP.NET WebForms already do a better job?”

I realized shortly after the Q&A was finished, when I had a chance to speak to them privately, that they were attending my talk to actively look for reasons why ASP.NET MVC shouldn’t replace WebForms at their place of work!

These developers were so afraid of trying something new and possibly failing that they actually went out of their way to research just the talking points they needed to not have to learn it!

This is an extreme example, but the point is the same: average developers cling to their favorite tools and stop learning new ones because they worry that leaving their pet languages and tools behind invalidates their years of experience.

Strong developers embrace the challenge and opportunity to learn new ways of doing things.

You won’t push your commits until the feature is “done.” (It’s never done!)

We call this “commit hoarding” at MarkedUp. This happens with developers who don’t have the confidence to tolerate criticism or scrutiny from the rest of the development team try to hide their work until it’s in a “finished” state.

Ultimately these developers hurt the productivity of the rest of the team, because we lose visibility into their daily output and they tend not to ask for help in the course of regular development. They’re responsible both for delays in shipping and for lots of last-minute bugs.

Strong developers recognize that they are not their code, and thus will put code out in front of the rest of the team more frequently and take criticism and suggestions with dignity and confidence.

You know just enough to be dangerous.

In an age where it’s become a lot easier for developers to work with threads, concurrent programming, distributed databases, and so forth you run into a lot of devs who can talk a good game. They can tell you all about thread pools, a hot new design pattern, or why a headless database is better than a masterful one in some scenarios.

They are able to know what something is and its potential benefits, but they don’t understand it.

I’ll give you a recent example: in C# 5.0 Microsoft recently introduced the async and await keywords, which make it radically easier to create and manage asynchronous callbacks. These keywords have even lead to the rise of some exciting new concurrent programming patterns like the TAP pattern.

However, do you know what this keyword has also lead to? A ton of average developers who suddenly think they can harness the awesome power of multithreading simply by wrapping every I/O call into a Task object in C#. They do this blindly, without any thought as to the costs of context-switching, multi-thread access to shared resources, and more.

And this creates dangerous, unpredictable, and very-difficult-to-test code in the hands of average developers who blindly uses this new toy everywhere.

Good developers not only know what something does, but they seek to understand why it was done this way and under what conditions it should be used.

Analysis paralysis.

Gandalfing the Documentation

Sometimes you’ll hear a member of our development team say “dude, stop Gandalfing the documentation and start coding something,” referring to the scene from The Lord of the Rings depicted above.

Analysis paralysis is a classic problem that affects lots of average developers; it’s often a problem of over-analysis, but in my experience the root cause is usually fear of making a bad decision.

Research is easy – implementations are hard! As a developer, you don’t want to make a mistake when it comes time to do the actual work. So just research the issue until the heat death of the universe and you’ll make a perfect decision for which you won’t ever be criticized!

Average developers worry about looking bad or making a mistake – they want to get it right on the first try.

Strong developers who are in unfamiliar territory don’t care – they’ll write shit code, unit test it, and throw it away in 45 minutes if they don’t think it can get the job done. Strong developers actively limit the amount of time they spend researching, because they recognize that it’s a trap – it feels like work but often isn’t.

You don’t invest in your tools or development process.

Average developers will spend a lot of time learning how some fancy new language feature or API works. They’ll invest a ton of time mastering the minutia of the Windows runtime or some new D3-based charting framework, but at the end of the day that doesn’t change the fact that they’re still average developers who churn out mediocre work at a glacial pace.

If you want to be a talented developer, you have to invest time into improving your skills and knowledge, and the number one factor that separates the men from the boys in most software markets is the ability to turn out production-grade code quickly. You can have both good code and speed – but you have to invest in your process you use for building it first!

Average developers don’t invest in their tools, their process, or their environment.

Think about it – when was the last time you took a step back and thought about the way you test your code or how you look for bugs? Have you tried experimenting with different types of abstractions for the same problems and seen which one produces more maintainable code? Have you tried picking up a new IDE or text editor to see if it can help speed up common tasks? What about investing in build or deployment automation?

Often times, the biggest improvement you can get as a developer isn’t focusing on improving the code you write – it’s optimizing your process for writing it.

Free protip for anyone who follows my blog regularly: the biggest improvement I’ve made in my own development process easily is decreasing my usage of interface-driven design and dependency injection. I do a lot more work using static classes and compositions these days and it results in radically less code to test and maintain.

You’re too embarrassed to ask for help.

When we started digging into the guts of Hive for some of our work at MarkedUp, we got a little stuck on how to tackle some problems we had specific to our use case. So what did we do? We reached out to one of the guys who wrote the O’Reilly book on Hive and asked him over LinkedIn. And he replied back!

We try to be self-sufficient developers to the furthest extent possible, but we also know when it’s more economical to simply ask someone for a favor versus beating our heads against our desks for hours on end.

Average developers get embarrassed and don’t want to reveal their ignorance, so they pretend they know what’s up until they commit some sort of horrible fuck-up in the codebase itself.

There is nothing wrong with saying “I don’t know how to do this.” Strong developers know this, so they go and ask for help when they’re stuck.

You don’t know how to make it easy for other programmers to work with your code.

Like an only child growing up, you never learned how to share your toys with anyone else. The same goes for your code – you developed some bad habits and an over-abstracted style that is impossible for other developers to follow.

An important part of working on any technical team is human parallelism – the ability for multiple people to be able to work simultaneously on the same codebase. But it’s also important for teams to be able to work asynchronously too – I should be able to make changes to your code when you’re away, and vice versa.

Average developers don’t think about their code this way – they simply set out to do a task and write it through to completion with the assumption being that they will always own that piece of code. Strong developers understand technical debt and try to limit it by designing code that is as maintainable and self-explainable as possible.

Writing readable code really requires developers to change their outlook – your code should last longer in the organization that you.

You don’t know how to read someone else’s code (or don’t want to.)

It’s an interesting paradox – average developers can’t handle a blank sheet of paper very well, but they can’t write the last 30 pages of a novel in progress either. Average developers are really hesitant to try to learn an existing codebase and often have to be handled all the way through it until they’re able to produce something.

A drastic example is when an average developer is brought in on a codebase written in a language / framework that they aren’t familiar with and will immediately want to rewrite it without a second thought to business value or time to market. It’s a type of pain avoidance – having to live with the precipitation of some other programmer.

Strong developers accept that the business costs in a rewrite are usually unacceptable and should be avoided, and they’ll diligently try to understand, learn, and modify the existing codebase sitting in front of them.

Reading code is harder than writing code, but strong developers invest time into learning how to excel at it.

You can’t code from the end-user’s point of view (you think too small.)

Average developers who manage to make it a lot closer to the “great programmer” end of the bell curve usually trip up here. You may have mastered some of the technical details and you may know how to work with other developers, but you’re still missing the most important piece: being able to see your code from the point of view of the real-world.

As a programmer, you job isn’t really to solve technical problems – you solve technical problems in order to solve business problems.

An average-to-bad developer will chase technical problems down deep rabbit holes without remembering why they were solving the problem in the first place. You’re not upgrading the version of MongoDB you’re using because there’s inherent value in upgrading – you’re upgrading because your team agreed that it was worth taking the risk to upgrade because of potential business upside due to new features / performance / reliability / whatever.

More offensively, average developers struggle and fail to produce anything of business value from scratch. When asked to design a new feature based off of a simple customer story average developers will take a rigid, literal interpretation of the story or spec and ship something that is barely useable by a human.

Average developers don’t anticipate other related use cases; don’t think about the end-user’s experience; and basically have to be managed heavily when working on anything user-facing. This is why so many of them get stuck writing line of business apps instead of products.

Great developers look at their code from the perspective of the end-user. How do I make this easier to solve my user’s problem? What are some other facets outside the literal content of the story that might make this feature even better for an end user?

If you’re a developer and don’t ask yourself these questions, then you’re not a great developer. You’re passable at best.

You aren’t able to judge the business value of any programming task.

Related to the previous issue – many technically strong developers fail to realize their potential due because they aren’t able to take a step back and look at their work from the perspective of the business or organization itself.

Strong developers are able to self-manage and make good business decisions about how they choose to invest their time. They’ll ask questions like:

Is this the most valuable thing I should be working on right now? How much time should I invest into doing this? Given the deliverables I have due in two weeks, is there something I can invest in right now that will make it a lot easier for me to hit that target?

Average developers don’t – they’ll take a spec and blindly implement it until it’s finished, without accounting for how their work relates to the company’s business goals and impacts other members of the team / other business groups.

The highest cost of any engineering team is development time, and average developers waste lots of it on minutia and technical tasks that are ultimately low business value.

Wrap up

If you want to be a better developer, it starts with changing the way you look at your code and how you program.

You have to understand and appreciate the business costs behind each line of code you write. You have to be able to step outside of your text editor or IDE and look at your work from the perspective of a customer or end-user.

You have to accept that your code will outlast you in any organization your work with, so design everything to be inherited by other developers one day.

Most importantly, never be afraid to take on new challenges and ask for help along the way. You won’t improve working alone in the corner on technologies you’re already familiar with. Software development is inherently social – learn how to bring other people into the experience.

If you enjoyed this post, make sure you subscribe to my RSS feed!



Be Your Own Measuring Stick

December 3, 2012 15:54 by Aaronontheweb in Startup // Tags: // Comments (1)

Today was one of those days when it was nearly 1pm before I was free to sit down and make my daily to-do list. There was water damage in my apartment, one of our awesome engineering candidates took an amazing job offer with a top video game studio instead of us, and I was way behind on all of the PR / marketing stuff I’m doing because we haven’t hired someone for the position yet. Typical, stressful day in the daily life of an early stage startup CEO.

So I sat down at the counter of a diner near our accelerator and started writing down my daily to-dos in the nearly-full, leather-bound Moleskine notebook I’ve used to keep myself and my thoughts organized for the past couple of years.

I just happened to open my notebook to an entry from 11/12/2011 (about a year ago) entitled “Skills I Want to Acquire.”

At the time I wrote this piece I was 15 months into my career at Microsoft and had just finished off a stressful pre-Thanksgiving crunch period, and I was starting to think about what I would need to learn in order to run a new startup.

I remembered how I felt when I wrote it – desperate to try my hand at starting a company for the second time1, but unprepared by my own standards.

I wrote this list as a roadmap to prepare myself for startup #2:

Skills I Want to Acquire – 11/12/2011

  • How to work with a team of developers
  • How to manage developers operations
  • How to manage a product lifecycle and iteration
  • How to measure customer engagement and how to incorporate those learnings into product design
  • How to build an online revenue channel
  • How to build a production + web and mobile service
  • Natural language processing + machine learning
  • How to build a developer platform / API
  • How to build a team
  • How to quickly prototype ideas and test them in market
  • How to manage a team
  • How to raise capital
  • How to acquire early customers
  • How to do great front-end development
  • How to build a brand

To be clear: these are all things that terrified me.

These are things I didn’t know or think I would be good at, but I knew I needed to learn them in order to realize my goals.

Today I went through this list and made the following notes:

Skills I Want to Acquire – 11/12/2011 updated 12/3/2012

  • How to work with a team of developers – done
  • How to manage developers operations – done
  • How to manage a product lifecycle and iteration - done
  • How to measure customer engagement and how to incorporate those learnings into product design - done
  • How to build an online revenue channel – done
  • How to build a production + web and mobile service – done, at scale
  • Natural language processing + machine learning – not done
  • How to build a developer platform / API – done, at scale
  • How to build a team – done
  • How to quickly prototype ideas and test them in market - done
  • How to manage a team – done; will be a lifelong process
  • How to raise capital – done
  • How to acquire early customers – done
  • How to do great front-end development – in-process
  • How to build a brand – done, but different than I expected.

To my immense pleasure, I had learned how to do most of these very scary, unfamiliar things in just a year! Of course I haven’t mastered them all yet, but I’m at a point where I feel confident that I can do any of these things competently at any time for our company and our team.

Given my day to day responsibilities at MarkedUp, I can’t imagine a world where I don’t know how to do most of the things on this list; it’s remarkable to think just a year ago that all of this stuff was alien and frightening to me.

Seeing this list and thinking about the progress I made from a year ago made instantly lowered my stress level and reminded me just how much progress one can make in a relatively small amount of time. It made me forget about all of the things I tell myself I’m doing wrong every day.

You are the ultimate judge of your own success, and I judge myself as having a really successful year so far.

If you want to do something hard like start your own company, start by sitting down and making a list of the things you need to do in order to get there, start trying to do them, and then review your progress against goal. You might be really surprised with how far you can get in a short period of time!

Now I’m going to get back to work on the next set of things I know I need to learn…

If you enjoyed this post, make sure you subscribe to my RSS feed!



Get a Grip

November 20, 2012 20:25 by Aaronontheweb in Startup // Tags: // Comments (0)

My regular source of entrepreneurial catharsis is watching Deadliest Catch.

If you've never seen it, it's a Discovery Channel show that follows four-six actual fishing vessels during two different Alaskan Crab fishing seasons a year (King and Snow.) “Bering Sea fisherman” is considered to be the most dangerous occupation in North America, hence the name.

Many of the crab boat captains are entrepreneurs themselves, being part or full owners of the boats. If the boat stops fishing, they go broke. If the boat breaks down, the fuel / parts / labor / time cost comes out of the crew and captain’s cut.

Deadliest Catch - fishing vessel coming down a 30-40 foot wave The captains are constantly put into situations that your average tech entrepreneur can relate to: dealing with emergencies, constant HR issues, having to make strategic bets on incomplete data, the emotional rollercoaster, you name it. The viewer watches the captains struggle with these decisions in every episode, and it’s as much of the experience and drama as the harsh nature of the Bering Sea itself.

What puts Deadliest Catch in perspective for me is this: people actually get maimed and even die on that show.

Boats get capsized by 60 foot rogue waves; deck-hands go overboard never to be found again during uneven seas; or get crushed by 1,200lb steel crab pots falling from a four-story tall stack. The fishermen’s lives, in addition to their livelihoods, are always at stake and are lost with regularity.

Back in the universe of technology entrepreneurship, the most danger I'm in everyday is getting carpal tunnel or eventually putting on weight due to poor diet and lack of exercise. No matter how hard things get, even if I fail epically, I’m not at risk of dying as a result of anything that can happen at any tech startup.

In one column: death, in another column: average health and with well above-average opportunities.

When I read maudlin stuff like today’s “The last day” short story on Pando Daily (with all due respect to Francisco Dao, a friend), I can’t really take it seriously.

I’ve failed a couple of times already in my short career thus far, including a big public flop in front of a number of my classmates when I was in college – and you know what?

You experience total humiliation and vulnerability when something you created falls apart, and that’s how you’re supposed to feel if you cared about what you did.

But dead startups aren’t things of life or death themselves – you’re not going to die; your stock will only rise as a result of what you did at your startup provided that you didn’t do anything unethical; most of your friends will admire you for having the balls to try something on your own in the first place; and you’re going to have a chance to try again.

It’s hard to ignore the self-critical things we all whisper to ourselves as entrepreneurs; they’re sometimes a healthy and necessary gut-check against reality, but more often than not they’re just self-destruction mistaken for self-motivation.

So don’t be so hard on yourselves and be the first in line to kick your own ass, because your market / customers / employees / investors / co-founders / competitors will all have plenty of chances to do it for you. Accept the occasional beating as an inevitability and recognize that the one thing you can always control is how you choose to react to things you cannot.

And now, I think it’s appropriate to share a little wisdom from Deadwood:

Deadwood: Al's Take on Misfortune

If you enjoyed this post, make sure you subscribe to my RSS feed!



5 Key Themes from Microsoft on the Future of Windows and WinRT from the //BUILD Keynote

I posted this originally on The MarkedUp Analytics Blog, going to put an excerpt here and then link you to the original post.

1. “Microsoft can only win by training consumers to expect consistent behavior, availability, and synchronized data across all of their different devices”

WinRT isn’t just about tablets – it’s also about fundamentally changing the way desktop software is consumed and unifying mobile / desktop / tablet and probably console apps all under one consolidated platform.

The unification of these platforms is the future of Microsoft; training consumers to expect consistent behavior and access to data across all of their devices is the only way Microsoft will be able to dethrone Apple and Google in mobile / tablet and protect themselves in desktop / console in the long-run.

Ultimately, Microsoft is really the only company that can execute well on native software, services, and devices. They are playing to their strengths (ecosystem and platform) and are doing it well here.

Click here to read the rest of “5 Key Themes from Microsoft on the Future of Windows and WinRT from the //BUILD Keynote” on the MarkedUp blog.

If you enjoyed this post, make sure you subscribe to my RSS feed!



New Features We’ve Added to MarkedUp: Custom Event Reporting and Reliable Crash Logging for Windows 8 and Windows Phone 8 Developers

MarkedUp - Analytics and Insights for Windows 8 DevelopersI haven’t posted much about my startup, MarkedUp, over the past couple of weeks on this blog (although I’ve been quite busy on the MarkedUp blog) so I figured it was time for an update on the company and product itself.

To recap, MarkedUp is an analytics service designed to help Windows 8 and Windows Phone 8 developers manage and monetize their Metro applications in the Windows Store.

MarkedUp is the only analytics platform available for WinRT and Windows 8 developers that supports full compatability with both WinRT and WinJS.

So what have we been up to since we launched MarkedUp back in late September?

Other than bug fixing, collecting user feedback, and acquiring users – we went to some lengths to add a couple of important new features to MarkedUp, both of which we released this week.

New Feature: Tracking and Measuring Custom Session Events

By far the most frequently requested feature among our early beta testers, this is the one that we’re most excited about: MarkedUp’s ability to track custom events and actions unique to your application.

how to use MarkedUp to track custom session events for your Windows 8 applications 

Custom session events are something you’d want to use whenever you need to track a metric specific to your app.

Let’s suppose you’ve shipped a video game – don’t you want to know how many games or levels a user plays each time they start the application?

Or maybe you created a content-reading application like my Lolcats Professional Pro Plus application for Windows Phone 7 – don’t you want to know how many pieces of content a user consumes on average?

Custom events can be used to track both of these – simply call MarkedUp’s SessionEvent method wherever something you want to track in your Windows 8 application occurs!

New Feature: Capturing Crash Logs and Unhandled Exceptions

The Windows Store gives you the ability to look through crash dumps, all in one inconvenient, giant dump file that you have to parse yourself.

We are a little dissatisfied with that approach to crash logging and analysis, and so are many of MarkedUp’s beta users. Thus, MarkedUp now has the capability to log crashes and analyze them via our convenient reporting interface.

how to use MarkedUp to log and analyze crashes for WinJS and WinRT applications

Logging crashes using MarkedUp is really easy to set up, and we describe how to use MarkedUp to log crashes and unhanlded exceptions in WinRT and WinJS here.

How to Get MarkedUp Analytics for Windows 8 for Free

Right now MarkedUp is completely free while in beta, but we’re restricting access to the platform through the use of registration codes.

However, I think it’s about time I gave everyone who reads my blog some love.

If you want to register for MarkedUp Analytics for Windows 8 for free, click here and use the following registration code: AARONONTHEWEB

If you enjoyed this post, make sure you subscribe to my RSS feed!



Search

About

My name is Aaron, I'm an entrepreneur and a .NET developer who develops web, cloud, and mobile applications.

I left Microsoft recently to start my own company, MarkedUp - we provide analytics for desktop developers, focusing initially on Windows 8 developers.

You can find me on Twitter or on Github!

Recent Comments

Comment RSS

Sign in