Architecture Review Meetings

29 December 2010

One technique I picked up from a colleague that I adopted on my most recent project, was having regular architecture review sessions. We ended up having these twice a week and it was a forum where we could review designs as a team. We were able to extend these sessions to include people like the security and infrastructure architects as well as the design lead for the project.

Being a fairly disorganised person we ran these sessions in the early stages as a free-flowing meeting more as a general discussion. As we progressed to align to project schedules and all that good stuff we were a bit more organised and had an agenda and had a minute taker to capture all those juicy bits of feedback that were raised in these sessions.

I found these sessions to be beneficial for the team firstly we developed a behaviour in the team of bringing designs in the early stages of forming to these sessions. With the varied experiences of everyone in the room there was some great alternatives presented that ultimately helped us come up with a better design. This behaviour was also good in that the team was happy to bounce ideas around in these forums, people have a tendency to take feedback personally but I think we got to the point where the feedback was constructive and in some cases was intentionally sought by the author.

Given the size of the team these sessions also provided a great forum for the team to get visibility of parts of the design being completed by others. If we had not had these sessions members of the team would have known their parts of the systems intimately, but not had a good end-to-end knowledge of the design. As we had a broad audience outside of just the architects, over time as we reviewed all of the components of the design there was a good history built up about how we go to the final design and the interconnectedness of all of the components.

The way the project unfolded people were assigned areas of the design more to meet the schedule than any natural strengths around domains. These sessions allowed the collective knowledge of the team to be harnessed to review the design. So although someone who had a history using ETL tools may not have been designing that component, within the Architecture Review meetings they were able to impart their knowledge on that part of the architecture.

Having picked this up and applied it to my first project it is definitely a habit to instil into a project of any size. Look for that balance between a free-flowing discussion and a regimented meeting that runs to an agenda. Get that balance right and you create a great forum where the behaviour of the team as a whole can be fostered to build a high performing collaborative team.


Twas the night before Christmas

24 December 2010

Twas the night before Christmas, when all through the office
Not a creature was stirring, not even a Tester.
The archietcture was drawn on the whiteboard with care,
In hopes that a Shaping Arhcitect soon would be there.

The Business Anlaysts were nestled all snug in workshops,
While visions of requirements danced in their heads.
And the PM in MS Project, and I in Visio,
Had just settled our brains for a replanning session.

When out of the lift there arose such a clatter,
I sprang from the meeting room to see what was the matter.
Away to the whiteboard I flew like a flash,
Tore open a Marker and threw up the flipchart.

The flouresent light on the new-drawn design
Gave the lustre of decoupling to objects below.
When, what to my wondering eyes should appear,
But a Business Sponsor, and eight tinny SMEs.

With a little old driver, so lively and quick,
I knew in a moment it must be the General Manager.
More rapid than eagles his coursers they came,
And he whistled, and shouted, and called them by name!


He sprang to the lift, and to his team gave a whistle,
And away they all flew like the down of a thistle.
But I heard him exclaim, ‘ere the lift door closed,
“Happy Christmas to all, and to all a good-night!”

A migration is much more than ETL

24 December 2010

From a business perspective a migration isn’t just a technical extract – transform – load process there is a lot more to it than that.

The business is usually undertaking a migration for a particular reason; consolidate technology platforms due to acquisitions or such or replacing an existing platform with a new better platform. In either case how you move the data can have a large bearing on the success or failure of the migration process. Sitting down with the business and talking through how a thing in the old world is going to look in the new world and what business rules should be applied to transform the old world thing into the new world thing is hard. Partly it is hard because you can’t see it is generally an abstract discussion which isn’t generally the most successful of conversations to have.

So if you can get someone who knows the current systems at a good level of detail and someone who knows the new systems at a good level of detail you may be able to evolve the discussion to come up with the transformation rules for the migration. One by-product of this discussion will be something I have seen referred to as overs and unders. An over is a field that exists in the old systems that doesn’t exist in the new systems; is it redundant, did we miss something when we specified the new system. Then unders are fields that exist in the new systems but we can’t find a value in the old systems, this is generally some new functionality that will need a default value.

Then I need to extract al the data out of the old systems transform it based on the rules you have given me above and then load it all into the new systems. Sounds simple, except you have a 24×7 business, oops. Now comes the trade-off between adding scope by having no or minimal disruption to the business and the associated cost in time and money to do this.

This is where the fun starts and from an architecture perspective here are some considerations:

  • Do the old systems need to be aware that the migration is occurring to cache changes and the like?
  • How do I extract the data so the old systems can remain online?
  • If the old systems are running once I have taken the extract how to I apply the delta changes to the new systems?
  • How do I cut over from the old system to the new system with no/minimal outage?

One other complexity that you may have to deal with is where your implementation strategy isn’t big bang but a sequence of iterative migrations. The complexity then is that you have to load all of the data into the new systems while they are still running phew!

So a migration is just ETL but the context within which you run the ETL changes the nature of it. This is what I refer to as the configuration of the ETL. So be aware that moving the data is the easy bit working with the business to understand the context that you will run it in is a key part of a successful migration.

Reference Architecture

23 December 2010

I have had the pleasure of leading a team of architects recently on a migration project. What seemed like a small problem but quickly became a real leadership challenge was dividing up an architecture across multiple architects. I have been on projects with multiple architects in the past but this was my first time leading a team.

In the past on projects where we have had multiple architects, we have used one of the following ways of dividing up the work:

  • Assign a domain to each architect and write an architecture document for each domain
  • Have multiple people working on the one architecture document

For this project we ended up going down a slightly different path and came up with something that I will definitely put in my back pocket and use again. To start the process off we came up with what we referred to as the reference architecture. This was a high level component based view of the architecture. The example of a reference architecture below was evolved by an architect taking a component or service and doing the next level of design for the functions identified.

Despite the reference architecture giving us a great high level component based view of the architecture one of the challenges it presented was providing guidance on what each components did and didn’t do. We spent a fair bit of time putting together a pack that went with the reference architecture diagram to outline for each component what was in scope for it and what was out of scope for it. Despite out best effort we still ended up having quite a few discussions about areas of the architecture and what component was responsible for this. Just something to be aware of if you follow this approach.

Once we had the reference architecture in place we were able to assign services and components to individual architects. To go with this we produced a light architecture document that was completed by the architect and then formed the basis of a review process. The document captured:

  • Problem Statement (from the pack described earlier)
  • Context (Highlight the component on the reference architecture)
  • Issues and Risks
  • Logical View
  • Integration View
  • Dependencies
  • Alternative designs considered

Once all of these light architecture documents were completed the content of all of them was cut and pasted into the central architecture document for the project. This was a relatively painless process with things like standardised modelling and content making it much easier.

This process evolved but looking back on it, it definitely provides a different approach to firstly developing the architecture and secondly managing the workload of a team of architects.

Migration is just ETL

22 December 2010

Over the last 9 months I have been working on a migration project which has been a new domain for me. Coming into the project I spent a fair bit of time trying to understand the domain to translate that into an architecture. Over time I came back to the simple answer that it is Extract – Transform – Load and I have ended up with that as the architecture. Where the ambiguity comes into it is knowing what systems you are extracting data from and the systems that you are loading the data into.

The roadblock that I hit was as an architect was that I wanted to identify the target and source systems that would form the solutions. To answer this question it became fairly clear that we would not know this answer until we had done all of the mapping from source to target. From a SDLC perspective this is in effect the detailed requirements/design which is way after the architecting bit.

Unfortunately we were in a situation where the target systems were being designed while we were designing the migration so were hitting dependencies again and again that we were trying to map to something where the design hadn’t been locked down. As a result of this, the mantra came about that as a project we were “special” and I really do think that we were but from a project management perspective there was an expectation that the normal waterfall SDLC would be followed and done in parallel to the design of the target systems. If I had my time again this is one issue that we should have driven a little harder to get a more logical alignment to the target systems design. The other solution could have been to use an iterative SDLC but this had its own issues being an organisation to great at doing iterative.

Architecting the migration capability, the main Non-Functional question that we needed to address in the design was how long it would take to perform the migration. There were several approaches to this but once again the problem of not really knowing the answer to the problem until we got closer to the finish came into play. We did try and look at other examples that we could use as a baseline which sounds OK in theory but in practice turned out to be a little more difficult. The technology is one part of a migration i.e. moving X GB of data from point A to point B and transform it on the way using a ETL tool. Then there are the operational components, things like reconcilliation and excpetion management, that add a variable amount of time to the migration. Talking to others who had done migrations the operation part is huge the first time you test the process and progressively become more efficient.

The answer around performance really boiled down to having the project in a position to do some sort of a trial run as early as possible and to performance benchmarks at this point. From a design perspective we needed to know what parts of the design could be tuned to deliver increased performance this included scaling the infrastructure and pre-migrating some data.

So a migration is just an ETL the complexities come about in that the details that drive the tuning of this design to allow identification of the source systems and also to meet the performance requirements. Move past the ETL problem there are more than enough tools to solve this problem focus on the bigger ticket items. But be prepared to call out dependencies on requirements that are needed to solve these other problems and look at ways to iteratively design the solution as these requirements come about, as waterfall just won’t do it.

1 Million Steps

4 August 2010


Great explanation of a TeamRoom

15 June 2010

Having had the opportunity a couple of times to be in a TeamRoom situation that was able to help a team become agile I found this description of a TeamRoom great. Good Luck in setting up your own room, but remember the saying you can take a horse to water but you can’t make him drink.

Quote: John Wooden

8 June 2010

If you don’t have time to do it right, when will you have time to do it over?

John Wooden

Global Corporate Challenge Day 1

20 May 2010

Well I have decided to do the Global Corporate Challenge again having done it two years ago. I have just got to work and the trusty pedometer is reading 2183 steps.

Good Luck to everyone else doing the GCC this year.


30 April 2010

This is so cool but so sad at the same time and you could make it sound like the real thing as well.