A Change of Screenery

I am moving house – Blogger has been home to many of my contemporaries for a long time and, like today’s teenagers, I am caving in to their peer pressure.

All future posts shall be here.

I never thought you were that bad Windows Live…

agile + scrum + timeboxing

I am a really big fan of timeboxing at the moment.  That basic principle of looking at what you’ve got to do, deciding on a sensible block of time to allocate to it and when you’re at the end of that time asking “if we spend twice as long on this will the results be twice as good?”  If yes then repeat, if no then next task…

I define a ‘sensible’ period of time as long enough to make adequate progress on the task at hand but not so long that your pace, focus and concentration drops off.  Typically you’ll be dealing in a handful of hours at a time.

As the title of this post suggests it obviously works really well in estimating sessions (close of those long, drawn out revisions of man days) and keeping energy and pace in stand-ups (stopping those meandering speakers) but I’ve also found it works well as a personal time planning discipline.  Writing presentations, holding roadmap discussions, making a dent in the email backlog and even editing these posts…

Advertisement

Excellent Software vs. Successful Software

It’s easy to get tied up in building the perfect application every time but is that always the right thing to do?  Well if you’re definition of ‘perfect’ is ‘exactly fit for purpose’ then I guess so; if you’re definition of perfect is ‘maximum quality in all areas’ then perhaps not.  You’re in a minority if you run a number of projects that are all equally beneficial to your organization; chances are your activities deliver various commercial and operational benefits over various periods of time.  That being so, should you build it all equally?

Let’s think about some examples – 2 different projects and some random software attributes (such as quality, performance, total cost of ownership).  How much should we invest in each attribute for each project?  Well what’s it worth to us…

Project 1 is building an embedded specialist site within our main sports trading site which is customized to take advantage of a very specific upcoming event.  We’re expecting it to be popular and bring in a lot of extra revenue in the lead-up to and during this event but after the event it is of no further use.  Where would we spend our time?  Perhaps we’d divide our focus a little like the radar chart below.  We have a narrow window of opportunity to squeeze value from it (the event and it’s lead-up) so we need it to be online all the time – so good quality.  We’ve said we expect it to be popular so lets make sure we build in enough capacity.  We know it’s a unique event and there is minimal chance that we’ll reuse it in the future so we can afford to skimp on the TCO and future flexibility.  It is temporary but it needs to be good while it’s here.

Project 2 is adding a content management system to another existing product giving marketing droids the ability to control the displayed banners, articles and links in real time.  It’s being launched in our product that needs the most cross-sell but if it works well we intend to roll it out to the rest of the suite.  What might that radar chart look like?  As below – we know we’ll be keeping this for a while so let’s spend more time on TCO (the longer you know you’ll keep something the more cost effectively you should try and make it run), we know we’ll extend it to the rest of the product suite if it’s successful so let’s keep some future flexibility in there.  We’re not processing cash through this thing so we can afford to take a bit more quality risk and given it’s an internal tool we know it has a limited pool of familiar users so ditto with performance and capacity.

It’s about working smarter – concentrating on the areas that make the most difference to the product and avoiding over-engineering where you wont get ROI.  Excellent software makes perfect technical sense but successful software makes perfect commercial sense.  Developing product from principles rather than hard rules or templates allows you to do the right thing in the right proportions.

Seniority vs. Experience

An interesting point I recently came up against in our hiring; we have a reasonably healthy balance in the team at the moment but, if anything, it’s a little light at the top end.  We thought we should be looking for more senior engineers but actually it turns out we’re really looking for more experienced engineers.

The difference?  A senior engineer is able to build very large, highly complex applications but an experienced one realises that this is a bad idea…

User Experience 101

Before we tuck into the last post in this series let’s have a quick recap of the 6 principles:

1. Scalability

2. Maintainability

3. Quality

4. Security

5. Availability

6. User Experience

I believe these things are the basics of building highly scalable online systems and as I’ve said before the basics are what every team succeeds or fails on.

User experience is a little different to the rest of our principles because it’s largely subjective – the first 5 are about hard facts and actions, things you can use and measure but user experience is about how your customers actually feel about your application when it’s all said and done.  Given our customers are who we’re actually building these things for in the first place you could argue this is the most important principle of them all.

Now let’s look at how we’d express this as a basic user story:

As the CTO, I want to represent products that are intuitive to use and feel fast and fresh to our customers, So that no matter where our users are and what their connectivity is they enjoy a high performance site, As proven by appropriate and justifiable answers to the user experience questions.

Like yesterday with availability notice our “so that” clause talks again about the considerations necessary for a global product.  Repetition is the building block of human learning so if it’s important to you, you can never mention it too often.  Appropriate and justifiable are interesting in this context, if your product isn’t usable people just wont use it.  How about betas and prototypes?  You’ve got to think carefully about what appropriate means here too, you can probably afford to spend less time on snappy performance and super quick load times but users are a fickle bunch so you’d better not skimp on the UI’s look and feel – you wouldn’t want a perfectly good product chucked out at prototype just because the human condition is to judge the kitchen by the dining room…

When you’re aiming for the best possible user experience ask yourself:

• Are user-facing error messages helpful and friendly?
• How snappy does my UI feel?
• How long does my page take to load?
• How does my feature feel on lower spec machines?
• How does my feature feel on low bandwidth connections?
• Is my UI intuitive and uncluttered?
• What do my end users think of my interface?

Get to know your customers and get to know what they like.

Availability 101

Speaking about availability, Live was down for a couple of hours on Friday when I went to post this – ace timing…

The next principle of building megaprise software is availability.  That one in particular is really important to us; some recent market research we commissioned concluded that a 1% increase in customer confidence in our site resulted in a 16% increase in ARPU (average revenue per user).  Simplified – people spend more when they’re surer you’ll be up and a 1% improvement isn’t out of anyone’s reach, especially when it pays back by a factor of 16.

So how would a user story for availability look?  Let’s start with:

As the CTO, I want to look out onto a resilient product suite that never stops even under catastrophic events, So that we never need to close our business and are always able to meet our users needs no matter what time zone they live in, As proven by appropriate and justifiable answers to the availability questions.

The time zone bit is an import part for us – we started off as a UK business, crept determinedly across Europe and with the recent inclusion of southern hemisphere markets I feel quite justified using the term global.  This has meant our traditional maintenance slots (the unsociable GMT hours) have evaporated and we’re left with the challenge of keeping trading going 24×7 releases and all.  Appropriate and justifiable?  Well put it this way, I want that 16%…

What can we ask ourselves to make sure we’ve thought about availability enough?  A good start would be:

• How will I make this resilient?
• How will this recover from failure?
• How will it behave when it loses connection to it’s data?
• How will it behave when its dependencies are missing?
• How will the whole system behave when my feature fails?
• How will this features status be monitored?
• What are the major events in the system and how do I make these visible?
• Are the error messages the events produce meaningful in diagnosis?
• How will the system survive the loss of a feature?
• How will the system survive the loss of a node?
• How will the system survive the loss of a data centre?
• Can the system recover from common failure scenarios automatically?
• What tolerance to network and system maintenance have I built in?
• Does my feature meet any SLAs imposed?

Plan for failure, after all it’s guaranteed.

Security 101

Continuing this week’s series about the six principles of webscale software we come to security.  This is a hot one for us as vulnerabilities can cost us serious cash, degrade our customers confidence in our business and irritate [understatement of the year] our regulators.  Simplified user story… familiar format… business value…

So what does our basic user story for security look like?  The sharper amongst us should be spotting a pattern by now:

As the CTO, I want to the peace of mind a highly secure suite of applications delivers, So that we continue to earn and keep our customers trust, As proven by appropriate and justifiable answers to the security questions.

And as the pattern dictates let’s think about what appropriate and justifiable means in the context of security.  Security is one of those things that’s easy to get carried away with as security professionals tend to be particularly good at raising awareness and we’re never short of threats to cringe in fear from (code flaws, viruses, internal fraud) and standards (ISO27001, SOX) to measure up to.  The one thing I’d add to this is to remember security is an investment just like anything else; you wouldn’t spend £20K insuring a £10K car and your products are no different.  Know the value of the asset, the risks it faces and the likelihood of eventuality – then you can make appropriate decisions.  Thinking about things like the industry you’re in (do you have regulatory requirements or other standards to live up to?), the data you keep (how confidential is it?), where the money is (do you keep any?) and how important reputation and those other intrinsic things are will keep you able to justify time spent on testing and securing software.

Some worthwhile things to consider as you code are:

• How could this component be misused and how have I prevented this?
• How have I limited this feature’s access to the rest of the system?
• Does the feature use the smallest possible data range?
• How might people use this feature other than intended?
• Is all private customer data securely transmitted?
• Where is this features use logged and is its activity easy to deduce?
• How will we be alerted to abuse of this feature?

Thinking about these things will help keep security commensurate with risk as you deliver features – tomorrow is availability day.

Quality 101

Yesterday was maintainability day and today I’m continuing through the basics of building software by tackling quality.  Quality is a massive, massive topic encompassing practices, unit tests, automation, measurement, environments etc etc and clearly we cant talk about all of that in any single post.  Lucky for us we’re talking about how you shape your thinking to make sure you’re always dealing with quality in the most appropriate way rather than any of these quality solutions.  Oh and let’s insert my usual comment about user stories and establishing the habit of thinking about the business value of an idea rather than just raw requirement…

Let’s cook up a user story for quality replete with “as proven by” clause:

As the CTO, I want to represent high quality software following a distributed and decoupled architecture, So that we can leave behind reactive work and our engineering activities can be solely focused on adding value, As proven by appropriate and justifiable answers to the quality questions.

So what does appropriate and justifiable mean to quality?  You need to consider a feature’s usage and the risk profile associated with it.  What’s the business impact of bug?  You will probably want to treat a feature that calculates the money a user is able to withdraw very stringently.  You may quite possibly be less worried if your content carousel stops cycling fresh promo material for a while – you can fix that when you’re good and ready.  Other things to think about are how many other people’s hands this code is likely to go through and what the feature’s planned shelf life is.

To help make sure you’ve covered your quality bases ask yourself:

• What is the anti-spec and how have I coded for it?
• Does my feature have tested and checked-in unit tests?
• How does my feature behave through a different user journey?
• Have I produced clear, readable documentation?
• Have I complied with any applicable design patterns?
• Do my comments and syntax make sense and match my documentation?
• Can I remove any lines of code?
• Will I be proud of this code if someone else reviews it?

If you can satisfy yourself with your answers to these questions you’ll be well on your way to reusable, supportable and easy to understand software.

Maintainability 101

Yesterday was scalability day and today I’m continuing through the basics of building software by tackling maintainability.  As we said yesterday organizations like ours find a really simplified agile user story a particularly effective way to communicate these principles; it’s a format people are already intimate with, practiced at delivering against and I personally like the fact that the basic clauses (as a… I want to… so that… as proven by…) force a habit of thinking about the business value of an idea rather than just raw requirement.

So how would a user story for maintainability look?  Building on how we decided to tackle the “as proven by” clause yesterday let’s throw it in right away:

As the CTO, I want to govern a set of products benefiting from continuously improving feature time to market and able to be fully maintained online, So that our delivery capacity becomes more consistent and efficient, As proven by appropriate and justifiable answers to the maintainability questions.

Again keeping in mind the keywords appropriate and justifiable; you’ll have to keep in mind the expected lifetime your application will have (yesterday we talked about seasonal features) and what it’s future roadmap is; how many more features are on the horizon?  How is it likely to be extended?  Are the windows for maintenance that do not affect revenue?  Not everything needs to be hot maintainable but you need to be able to modify it quickly and easily when you do.

These are the sorts of things do you need to ask yourself ask you work:

• How will I upgrade this hot?
• How will I add features hot?
• How will changes be rolled back without tattooing?
• How can I run multiple versions of my feature concurrently?
• How have I isolated the feature to limit the scope of future changes?
• How have I limited the manual effort required to deploy?
• How will I add or reduce capacity online?
• Is all my functionality exposed via an API?
• How have I prepared my feature for quicker future time to market?

Having sensible answers to these questions will help make sure you’ve prepared your product the best way possible for future development.

Scalability 101

Last week I posted about the basics of building megaprise software and the six principles I believe should be reflected in every product you deliver [if you’re in this category].  For organizations like ours a good to describe each of these principles is a really simplified agile user story; it’s a format people are already intimate with, practiced at delivering against and I personally like the fact that the basic clauses (as a… I want to… so that… as proven by…) force a habit of thinking about the business value of an idea rather than just raw requirement.

So how would a user story for scalability look?  Let’s start with:

As the CTO, I want to look out onto an infinitely scalable set of products that can be grown cost effectively with customer demand, So that system capacity never restricts business growth.

Hmmm – how do we cater for the “as proven by” clause?  When developing a feature from a user story this is one of the most important clauses as it is the primary reference you use to know if you’ve done a good job or not – but how can you tell if you’ve done a good job of scalability?  The hard data (design, performance thresholds and capacity) will be different in every system, feature and project so how about taking the principle a little further with some questions you can ask yourself – essentially thinking exercises you can use to judge how well you’ve reflected the principle of scalability in every feature you roll.

That would give us:

As the CTO, I want to look out onto an infinitely scalable set of products that can be grown cost effectively with customer demand, So that system capacity never restricts business growth, As proven by appropriate and justifiable answers to the scalability questions.

Note the keywords here; appropriate and justifiable.  You’ve got to keep in mind that there might not be business value in making every single feature you roll infinitely scalable – what if it is a temporary feature used only for a fixed time like a migration tool or a site aimed to take advantage of a single event like an election or sports championship?  In these cases you’ll probably have a fixed demand for capacity and a finite lifetime all known in advance.  Keep it appropriate.  A good practice to help make sure you always think of the most appropriate solution is to practice explaining it to yourself.  If you can’t convince yourself you’ve catered for it then you’ll never convince me and you will most certainly never convince your customers.  Keep it justifiable.

What do you challenge yourself with?  What questions do you need to answer?  A good start would be:

• How will I scale this again and again?
• What is the marginal cost of increasing capacity?
• Will it scale cost effectively?
• Does it perform adequately?
• What will happen if it gets hit 1000 times per second instead of 10?
• What will be the effect on shared data/infrastructure if its usage doubles?
• How do we measure usage and remaining capacity?
• Can I do this in fewer hits?
• Can I do this with less memory?
• Can my feature deliver its functionality in fewer cycles?

Think about these things as you code and you’ll be well on your way to making highly scalable software.