Is Architecture still Relevant when we are Agile; Moving Everything to Cloud; Going AI?

You still need to:

• know where you are and what you have and what state it is in

• know what you want (at least at a conceptual / business level)

• know what you need to move from current to future

• guide choices, implementation and make tradeoffs between paths of action

• have relevant data, information and content to deliver the products and services and make decisions

• evolve your application and technology landscapes to take advantage of developments while managing risk, security, privacy and efficiency concerns

• partition work across teams in a way that allows their results to be integrated

• have a realistic roadmap to manage expectations across customers, business and technical teams

• manage quality of implemented solutions to ensure service, customer experience and future maintainability

Of course, properly chosen and employed, AI can help with many of these!

Models for Transfer of Perception

Research on physiology of the visual system, neuroscience and cognition shows our perceptions are controlled hallucinations based on predictions, progressively refined by feedback of sensory input.

Perception starts, not with sensory input from the eyes, but with a prediction in the brain. We then look for confirmation of our prediction in the sensory input. Where we detect anomalies, the brain adjusts the prediction and seeks new evidence. This has profound implications for models to share our perceptions.

First, we need to consider the receiver’s world view & experience and their likely predictions, which affect how they interpret what we share. If we can translate our message to a medium, format and notation that corresponds to what they are familiar with, we are far more likely to succeed. For a database designer, using UML diagrams is a good strategy. For a risk manager, this will fail, but they may be receptive to a diagram that categorises information according to risk parameters such as sensitivity to exposure and impact of exposure. It will help if we prime the expectation (and hence the prediction) with defining shared concepts. Ontologies are valuable in providing the nouns for later communication.

Enterprise Architecture benefits from a strong meta model defining the agreed concepts, relationships and properties we deem important that describe instances of the concepts. The meta model and appropriate tools (repository and modeling, query, reporting and visualisation) allow us to collaborate, to share and support multiple formats, notations and frameworks to work successfully with different communities.

The theory of communication (Shannon and Weaver) advocates redundant information in the message and feedback to the sender to ensure that messages were successfully transferred. For visual models, this translates to providing “dual coding” i.e. both symbols and text to facilitate correct perception of the information. We should also query the receiver to ensure they perceive what we intended to convey.

Efficiency can be enhanced using principles from compression of messages (e.g. in transmission of images, audio or video), where prediction is also used to predict what comes next and then to only transmit information if the actual signal differs. E.g. in a video we only transmit the changes from frame to frame, a very small proportion of the full image of the frame. For models, this translates to providing only difference or exception models, or models where the important information is highlighted in some way, by colour, modifying symbols or other means. In this way we have a lot less to encode and interpret. Again, competent tools can assist by providing filters, query, summary and highlighting mechanisms to generate useful outputs, potentially “on the fly”.

We are researching these topics and progressively enhancing our tools and methods at Inspired.org. Let’s connect!

Quality in Architecture

Architecture is about change. Not change for change sake. Real change that produces value and leads to Desirable Futures that enhance delivery of value to all stakeholders, incl. customers, employees, shareholders, business partners and society at large. Futures that enhance people’s lives. To deliver that change, what we produce (including the internal ones) have to be delivered at quality. That said, it is surprising how few architects understand Quality Management. This post will explore a few fundamentals.

Quality is not a vague “goodness” it is defined as Conformance to Requirements. OK, but whose requirements? Well, everyone affected. E.g. If we are introducing a new product or service this will include: customers, service personnel, business partners, the organisation (represented from a financial perspective by the CFO), other board members with responsibilities such as compliance, return on investment), regulators, implementors (who design, build, test and deploy the product) and support and maintenance personnel. Requirements include functional aspects: “what must it do?” as well as non-functional aspects: “How should it do it?”. E.g. function: “Generate Energy” and non-functional: “With minimal noise and pollution & at least as efficient as existing options”.

The performance standard is Zero Defects. This means meeting requirements, not perfection. E.g. A power generation service should produce energy 99.99% of the time and be recoverable in < 1hr if it does go down. Provided it performs within these parameters, it is considered “Zero Defect”.

The System of Quality should be Prevention rather than Appraisal. The latter focusses on inspections at the end of the lifecycle to stop bad product/service going out the door. This is necessary, but not sufficient. We need to track down root causes of deviations and eliminate them. This is prevention. Using prevention, we only spend on correcting a given type of error once, reaping recurring benefits on every instance of the process / product / service delivered. Spend on prevention yields continuous improvement. The rate of spend determines the rate of improvement.

The Price of Quality is measured as the Price of Conformance plus the Price of Non-Conformance. The former includes all costs to ensure we do the best job and make it Right First Time. Examples include training, methods, standards, inspections, tools etc. The latter includes all costs incurred because something was not right. Examples are: waste of materials, waste of time, financial loss, consequential damages, loss of reputation etc.

Using prevention, the Cost of Quality is lowest at Zero Defects. This is great news and gives us the empirical rationale for allowing and doing great work. Proof is the remarkable 2-3x improvements in productivity realised by organisations like Hitachi Software and Computer Sciences Corp by focussing on quality and eliminating errors early, i.e. in requirements.

Positive Risk? Should we take more risk? If so, why?

Watching Rory Sutherland, I saw an example where avoiding a small risk eliminated a major innovation opportunity.

As a one-man plumber, we must do each job well and follow proven best practices to avoid reputational damage and future business harm. The cost of a botched job is significant and may harm our cash flow. We’re likely risk-averse.

As a larger plumbing enterprise, we can spread risk by allowing one plumber to try a new approach. If the job fails, it won’t significantly impact our cash flow since other plumbers are operating normally. Even if we have an unhappy client, the other 19/20 or 95% will still recommend us, so our reputation is high. If we address the issue, we can restore 100% satisfaction.

By spreading risk across the organization, we can make it relatively small. If the new approach is successful, it can save us 20% effort on similar jobs, increasing our margin or profit and recurring over time.

We can divide risk (limited to one job and one client) while multiplying benefits by team size and job frequency. However, we must define success (e.g., client satisfaction and time or resource savings) and measure these. If the innovation fails, we must stop doing it.

If we want to limit the downside further, we can partner with the client where we intend to try it. Many will accept some risk if we pass on the savings or benefit. They’ll also tolerate problems or glitches more happily if they’re informed upfront.

Taking this into account, we need to be more tolerant of taking controlled risks and bringing in innovation, especially in large organizations.

The graphic shows a concept we introduced two decades ago for bringing innovation into an organization in a controlled way. A small number of innovators try out new things that may be beneficial. For those that are, they create proofs of concept and do some organizational learning, distilling how we can usefully use the technology or approach. The methods group translates their learnings into how these would be applied in the organization at scale and trains the operational teams in how to apply them. The operational teams apply what they’re taught and measure the results, feeding back to the methods group. This way, we exploit the effects of innovation - large infrequent gains, sometimes through radical change, while limiting the negative effect of taking the whole organization through a learning exercise and drop in productivity. The cycle of improvement, application, measurement, and improvement between the operational and methods groups exploits the principles of Kaizen, viz. small continuous incremental improvements, leading to high quality and efficiency.

Agile Process or Artefact?

There is much hype around Agile these days. We know that things are moving faster than ever and that “slow business is no business”.

There are agile methods, agile techniques, agile projects, agile best practices, agile coaches and agile certifications. But how many agile businesses are there?

I support the original tenets of the Agile Manifesto and the teachings of the founders. They highlight user involvement, refinement through iteration, communication, flexible working and trust relationships between business and technologists. All good things.

I do have a problem with agile dogma and religion in the absence of other essentials for Agile practices to work. These essentials include: high skill, small teams, continuity of resources, trust and attention to quality and testing. Agile should also not be used as an excuse to bin architecture, requirements definition and documentation.

I often see Agile abused to “crank the handle harder” in development projects in an attempt by business to get to functional goals earlier. The emphasis here is wrong. We don’t need a more pressured process, we need a better result: higher quality (better matching requirements) and more adaptable to future requirements.

Using a building analogy, many Agile projects are “laying bricks faster”. This may be without fully understanding requirements, proper design for safety, scalability and still creating a “fixed” artefact, namely a brick and mortar building. If requirements change, as business evolution dictates they will, another project is required to break down elements and build others.

I suggest we should be building more flexible artefacts. Take a conference centre, which must host a sports match one day, a book fair the next, on the weekend a rock concert and the following week a motor show. We know it has to be flexible and that we won't have time to rebuild. Flexibility is thus a stable requirement. Knowing that, we can design and build for it, even using a conventional construction approach. We can build a shell with large open, reconfigurable spaces, movable walls, power outlets in the floors, lighting on tracks, etc. This will allow the user community to alter the building themselves in a matter of hours.

We need Architecture to identify high level stakeholder requirements and change dimensions and try out conceptual options rapidly, partition elements that can be tackled by separate teams with defined interfaces and then to guide implementation. The construction itself could be agile or conventional. The former offers advantages where some requirements and suitable solutions are not known and need to be tried, prototyped and refined. Using an Agile approach requires high skills, stable teams, continuity of resources as well as a high trust from sponsors.

The solutions should be documented so they are easily used by the operators and maintainable for adaptation.

#Agile #SolutionArchitecture #EnterpriseArchitecture

All Roads Lead to ROME

No, not the historic city. And in fact not all, but few! ROME here stands for Return on Modelling Effort, a term coined by Op 't Land, Proper, Waage, Cloo and Steghuis in Enterprise Architecture - Creating Value by Informed Governance. Springer, 2008.

It is important: organisations spend thousands of hours and millions in revenue building various kinds of models in strategy, finance, enterprise architecture, requirements, solution design and other areas. Often these are useful to their creators, but do not deliver organisational value in excess of their cost.
The problems are often related to appropriateness of models to the kinds of analysis required, accessibility of the models to relevant stakeholders and quality of the models themselves.

Appropriateness relates to the relevant concepts being included, the ability of the models to capture knowledge, share knowledge, support analysis and insights and serve as inputs to further steps. It is also influenced by the appropriate level of abstraction and detail.

Accessibility is related to the representation of the models in ways that are familiar to the relevant stakeholders, ease of sharing and distribution. Architects love complex graphical models, but a business executive might prefer a poster, powerpoint deck or even a story. A financial executive will prefer a spreadsheet or graph. We also need ways to deal with complexity and scale to make them manageable, as well as ways to highlight important insights, rather than overwhelm with volume.

Quality is related to accuracy of gathered information, mapping to the model, fidelity of the modelling techniques and accurate translation to accessible output formats.

Achieving high ROME depends upon a few fundamentals, including: clearly understanding the purpose of the models; identifying and knowing the frame of reference of the Stakeholders in both business and technical domains; choosing appropriate modelling concepts, techniques and representation language that is effective, concise, efficient and precise enough. It also requires sharing outputs (including interim or evolving ones) and engaging various stakeholders positively in the process of model formulation, improvement and evolution.

#ROME #Modelling #EnterpriseArchitecture #BusinessArchitecture #ModellingLanguage