Power your way to the crucial answers for your solution.
A Podcast with transcript and slides.
Today I am going to talk to you about the three most powerful architecture decisions.
The little backstory first: I transitioned into architecture in the year 2001. My first substantial project was a dealer incentive management system for a large company. There was so much to work out. It was daunting. Where would I begin? How would I know if I was making the right decisions? I had the formal classroom training, but you know how it is, unless one gets into the deep end, it’s all theory. I completed it somehow. But it was only after doing four or five architecture projects, that I started working things out more easily. I began to see the patterns in the problems and solutions. And I discovered efficient ways of getting good outcomes. Today, I will be sharing with you a technique I have refined that I hope will demystify a critical aspect of architecture for you – how we can expertly deal with architectural problems. It can seem mysterious to outsiders. But it is not an art, and it does not have to be a laborious struggle. We can make life easier for ourselves and deliver fine outcomes for our customers.
Now, if you look at a typical IT landscape of systems and how they work with each other, the complexity can be enormous. Imagine scores of systems and hundreds of interfaces. If something new or a significant change comes along that may affect a large area, how does one start dealing with the potentially large number of possible architecture decisions required? And how do we know if we have asked and answered the most important questions?
Here’s how we will slash a path through this jungle. We need to start with these tools.
- First, a clear destination: which is to say, clear functional and non-functional or quality requirements.
- Second, a birds-eye view of the land: meaning the details of the existing architecture in the form of lists of assets, their functions, integrations and diagrams.
- Third, a machete: that is a clear and sharp architectural thinking technique.
[This includes keeping Design Decisions (DDs) and Technology Decisions (TDs) separate from Architecture Decisions. How they differ is beyond the scope of this Podcast. Please message me personally if you want to know.]
Equipped with these tools, we can clear a path through the undergrowth in three stages, in the form of three types of Architecture Decisions (aka ADs). Why three steps and not five or ten? Well, this is my special insight. From getting through such jungles a few hundred times till now, I have learnt that these are the most critical problems and if we deal with them first, we make headway fast and get control over the other ADs, DDs and TDs. Trust me and try it. It works for me, and it will work for you.
So, what are these three fundamental ADs? They are:
1. Function Placement ADs
2. Data Source ADs
3. Integration Method ADs
Let’s have an in-depth look at each. Now keep this in mind, we are applying the core architectural motivation of achieving the maximum functionality and quality at the minimum long term cost. I call it the Science of the Optimum. The fundamental architectural principles that help to get this balance are called Cohesion, Loose Coupling and Reuse. They drive several services, data types and interfaces to the same component and separate others. [You can look up these three essential concepts in books or the internet; or drop me a line, and I’ll explain them.]
Function Placement (FP) ADs
They address the foremost problem – where should the functions be executed? To work this out, take the Functional Requirements one-by-one and see if there is an existing system, component or sub-component (these formal architectural constructs are the subject of a separate article), or a combination, that provides the underlying technical function. If not, see if there is one that you can extend to do it. No joy? Then you need a new architectural component for it. Reusing systems is the least expensive overall and creating a new one the most expensive and to be avoided if at all possible. And for a new one, buying may be cheaper today than building. Repeat this AD for every new or modified business requirement.
So, here is a mantra that you can apply while attacking Function Placement ADs: “Reuse before modify before buy before build”.
Data Source (DS) ADs
The second problem that we deal with is – ‘From where will the function get the external data it needs?’
Information Technology is essentially “data in – data out”. And the overall enterprise cost of IT is pegged to the amount of data it handles and stores. Storage costs per Petabyte are falling, but the need for the enterprise to reduce costs will always outpace everything. So, we architects have to take the utmost care to optimise the flows and storage of data.
There are two main types of data stores – those for Systems of Engagement (SoE) and those that are Systems of Record (SoR). SoE do not need to store data for very long, whereas SoR do that for regulations and analytics. Typically, SoR data stores can be ten to a hundred times larger than SoE data stores.
Now, for an application that needs to carry out a function that needs data that it does not itself generate or ‘own’, the architectural problem is whether it should take it from the external owner, let’s call it the ‘master’, or some other system that already has a copy of it, let’s call it the ‘record’ or make a ‘local’ copy for itself.
There has to be a really good reason for making a local copy of some other system’s core data because every copy involves computing and temporary storage costs at the source, network costs to transport it and computing and storage costs in the destination system. A few of the right reasons are – low latency requirements, information enrichment that is owned by the consuming application and a retention requirement that is longer than provided by the master. Take these factors into account while making this AD.
Now, here is the mantra that you can apply while attacking Data Source ADs: “Master before Record before Local”.
[I have a litmus test that quickly and objectively indicates whether we should use the external master/record data or a local copy. I want you to focus on the steps in this Podcast, so I will leave it out and publish it in a dedicated future article. You can drop me a line if you want it right away.]
Integration Method (IM) ADs
Having got so far with the Function and Data Source ADs, the edge of the jungle starts coming into sight and Integration architecture decisions will get us through. They are relatively straightforward, but not any less vital. The focus is to ensure simplicity in communication between the many components in the overall architecture as the cost here comes from maintaining the resulting mesh of connections. Direct connections between systems are the simplest, and it is possible in a mature Service Oriented Architecture. Here, well-designed clients and serving applications can take care of integration needs such as standard protocols, publish-subscribe patterns, peak load smoothening, data aggregation, etc. Only where this maturity is not available should we use Enterprise Application Integration (or EAI) platforms to provide the necessary facilities. Finally, an old fallback is to exchange the required information between applications using files. With file exchanges, there can be data reconciliation concerns and additional processing, network and storage costs. But for certain problems, it is a natural choice. It is suited for larger data sizes, higher latency toleration and batch operations and used more often even now than one may think.
Here is the mantra that you can apply while attacking Integration ADs: “Direct before EAI before files”.
[For this too, I have a litmus test for deciding between direct and EAI mediated integration, but like the one for data copying, I will leave it for a dedicated future article. Please contact me if you want it immediately.]
As you progress through the AD jungle, you need to put down a map of your way through. So, record the above ADs in a template. It should have the AD ID and Type, a concise problem statement, the solution options considered, the decision, the reasoning and any implications. Mention the related ADs, TDs and DDs as they become available, for cross-reference. Filling this template will help show others why you took a particular course, and they will be able to follow you easily.
[The technology and design of a new item in any type of AD should be dealt with after all the ADs are completed, as focused design and technology decision tasks, likely with the help of experts in respective areas. The same approach of problem->options->principled selection->explanation->implications->recording applies to them too.]
These three ADs of Function Placement, Data Source and Integration Method apply to the problems of all the architecture disciplines, but especially to business, enterprise and application architectures.
Of course, there are several more domains – enterprise, information, integration, infrastructure and security. To cover all the seven domains and their layers across the many IT systems could mean scores of problem statements and decisions.
But with the above approach, you will break the back of the work and deliver high value fast for your customers.
Enjoy, my friends.
Connect with me: 
Awesome and this is most useful.