Blog >
IT Architecture
Immutable MicroServices
I love the concept of immutable microservices. In short, it means that every component of a system is designed and packaged as a microservice and those components' configuration is never changed. This means, virtual servers, virtual switches, databases, applications, etc are all versioned and if a configuration change is required, a new copy of it is deployed on the correct version. For this to work, the organisation has to understand a few base patterns: |
Go live first IT methodology
When developing a new system, companies often view go-live as the last step in the software development life cycle. This causes a mental disconnect in people’s minds having on the one hand the SDLC of: inception, design, implementation, installation and on the other hand the production support. In the modern time this mindset is no longer efficient enough, with companies moving towards a DevOps and NoOps operational strategy instead. I’m proposing that regardless of which operational strategy companies use, they should also adopt a “Go live first” approach. By this I mean that any project to develop a new system or replacement of an existing system should begin by installing a placeholder for the system into production and holding a go-live event for that placeholder. This means, the system has not been developed, the business requirements are not yet collected. All that is available is the high level vision of what the system will accomplish and a placeholder. During the "go-live first" event the management backers of the project explain the vision of the project to the future users of the project and show them how to access the placeholder and how to contact support about the system. The business analysts then start writing business requirements and system architects write designs, but instead of writing it as if for a future system, they write it as changes to an existing system. The operational staff support the system as they do for any other production system. The benefit of this is that it will stop companies from building development support systems that are not used to support production system. It will coordinate all IT development into focusing on the productionisation of the systems and how to support it while it is being changed. It will encourage IT staff to be more closely aligned with business and it will enable business people to think and act with more agility. |
Software development as the translator in the man – machine communication model
Software development is the process of enabling users to
communicate instructions to a machine and have it perform those instructions as
expected. More generically, software development is the action of building a communication
channel between a user and a machine. This communication between user and machine closely follows the Berlo SMCR model of communication as explained through the following example: 1. Instruction a. Source – The user has an intention to have the machine perform an action. b. Message – The instruction which the user wish the machine to perform. c. Channel – The software accepts the message through a user interface and translates the message into the relevant format for the machine. d. Receiver – The machine receives the translated message and acts on it. 2. Feedback a. Source – The machine has succeeded or failed to perform the instruction. b. Message – The feedback on the execution of the instruction. c. Channel – The software translates the feedback and presents it to the user through a user interface. d. Receiver – The user understands the feedback presented. Note: Software systems may consist of a chain of channels, each being a receiver of a message and a source for the next link in the chain. As long as instructions originate from a user and gets executed by a machine, the above process holds for each link in the chain as well as the chain as a whole.
Software development is responsible for making this communication as effective as possible by limiting the translation failures (noise) at every step. To accomplish this, a software developer must optimize the following variables: 1. Understanding of the user's intentions and expectations 2. Understanding of the machines capabilities and language 3. Translation of the message into the machine's language
An enterprise system has many thousands of users and many hundreds of machines combined into one software system. This makes it near impossible for one person to optimize all 3 variables. |
Software design principle - Robustness
Robustness is probably one of the most important priciples in software development. Even more important than Security or Reliability. The reason is simple. If the software is not robust then it can not be secure nor reliable. Robustness indicates how well the system's other features perform their function and continue to perform their function. Robustness is a principle and not something you do, if you copy what someone else does to create robust code, you misssed the plot. Below are some articles on robustness, learn from their thinking process and not from their lists of things they do to improve robustness. Alberto Gutierrez has a good set of 10 characteristics to learn from. Matt Bishop's robust programming article. You may draw a robustness diagram to see all the potential fragile points in your system. Read more on how robustness is often overlooked or wrongly called "quality". iMatix tells how they build reliable/robust systems. |
Organization of the architect
Based on the concept, Organization of the artist devised by architect Frank Gehry. The organization of the artist places the architect/artist in control of the design throughout construction and deliberately eliminates the influence of politicians and business people on design. Many IT projects go over budget or fail due to the architect not being in a position to truly run the project. The architect must be mature enough to know the limit of his technology and development teams, he must also know how to get the true needs from the customer and not be swayed by the loud talkers and political players in the customer's ranks. |
CouchBase Memcache & CouchDB NoSQL
Brisk: Cassandra & Hadoop NoSQL mashup
DataStax’ Brisk is an enhanced open-source Apache Hadoop and Hive distribution that utilizes Apache Cassandra for many of its core services. Brisk provides integrated Hadoop MapReduce, Hive and job and task tracking capabilities, while providing an HDFS-compatible storage layer powered by the Cassandra DB. DataStax Brisk Apache Cassandra Hadoop A key benefit of DataStax’ Brisk is the tight feedback loop it allows between real-time applications and the real time analytics that follow. Traditionally, users would be forced to move data between systems via complex ETL processes, or perform both functions on the same system with the risk of one impacting the other. http://www.datastax.com/products/brisk http://ria101.wordpress.com/2010/02/24/hbase-vs-cassandra-why-we-moved/ http://www.theregister.co.uk/2011/03/23/cassandra_mashed_with_hadoop/ http://allthingshadoop.com/2010/04/24/running-hadoop-mapreduce-with-cassandra-nosql/ http://pig.apache.org/ Rackspace casandra by example: http://www.rackspace.com/cloud/blog/2010/05/12/cassandra-by-example/ |
Lean Software Development Plans for Change
The Kaizen way is to plan for change. In software development
this can be a bit of a problem because changing software is usually
exponentially more expensive than it was to write in the first place. To make software development lean, one must
counteract this exponential change curve. This is accomplished by using
refactoring, automated testing and continuous integration in order of priority. Once this is in place, change and quality becomes cheap. From my experience I cannot remember any projects where a heavy up front design has produced a working
project, I can however remember many projects where the principles of refactoring
has saved a failing project. The most successfull projects used refactoring as a way of coding, from the start new features are refactored into existence. For more information read Martin Fowler's excelent article on the same topic: |
Complex Event processing engine
CEP with DB4O http://www.db4o.com/s/cepdb.aspx http://developer.db4o.com/Solutions/ComplexEvents.aspx Esper is an open-source CEP engine written entirely in Java and fully embeddable into any Java process - custom, JEE, ESB, BPM, etc. It recently has reached version 2.0, is backed by EsperTech under a professional open source / dual license business model, and triggers a growing interest in the Java and .Net community as well with NEsper, its full .Net/C# implementation. Both can be downloaded from esper.codehaus.org with complete examples and documentation. The Esper engine works somewhat like a database turned upside-down. Instead of storing the data and running queries against stored data, Esper enables applications to store queries (or statements) and run the data through them. Response from the Esper engine is thus real-time when conditions occur that match queries. The execution model is thus continuous rather than only when a query is submitted. One of the most attractive features in Esper is that is easy to formulate continuous queries: Esper provides an SQL-like language extended to handle event streams and patterns, an Event Processing Language (EPL). For developers more inclined to program against an API, a continuous query object model API is also available. http://www.theserverside.com/news/1363826/Complex-Event-Processing-Made-Simple-Using-Esper |
1-10 of 13