Monthly Archives: November 2006

A Pattern Language for Versioning

Global Context: Software Change

You are releasing a piece of software (i.e., a software artifact) for others to (re)use as a building block in their software. Reusable code fragments could be delivered as libraries, software components, applications, services, etc. Others find your artifact useful and begin using it. They could use the functions or classes within your library; they assemble applications out of your component(s); they compose your application in their workflows; they build business processes that invoke your service(s).

Once others start reusing your software you discover that you need to modify it. There are instances where modification is not possible. For example, software controlling deep sea sensors or satellite equipment may be out of reach and thus impossible to change, even when its users would like to do so. However nowadays that is the exception rather than the norm. But why change software artifacts that others are using?

One common reason for software change is a bug fix. In spite of stringent testing procedures the probability of finding errors after the software passes quality assurance and ships is not zero. Consequently you (or those using your software) may discover an error in released software. After understanding what is causing the error you modify the code to correct it.

Another driver for change is evolution. The requirements change. For example, new legislation may mandate compliance with a standard. Alternatively there could be a change in the environment. For instance, one of the systems your software is integrating with is being phased out and replaced with a new one. The new systems have slightly different interfaces and your software must change to accommodate them.

Regardless of reason changing software could cause failures—an undesired effect—even when the change fixes an existing error. Consequently the value of the change is context-dependent. For some users the change critical: they cannot operate without it. For others the cost of accommodating the change may be higher than the cost of compensating for it: their software may be counting on the bugs being there, and removing them would entail significant work.

Software change and its impact on those using the software provide the shared context for this pattern language.

Non-breaking Change

Not all types of software change cause failure, having an undesired effect on its users. Sometimes nothing breaks following a change. Ideally all changes would non-breaking. In reality this is the exception rather than the rule, which makes this case uninteresting.

Breaking Change

You released a software artifact. After the release you change it to accommodate a bug fix or a new requirement. You release a new artifact that replaces the old one. However its users start having problems; clearly the shift from the old artifact to the new one is not a NON-BREAKING CHANGE.

Why would a software change cause failure?

A change breaks software when the new artifact (i.e., the software with the change applied) violates one explicit or implicit expectations that the users of, or other software that depends on it, have about the old artifact.

Therefore:

A change that violates an expectation about a software artifact impacts its dependents. User interacting with the artifact may experience unexpected behavior. Likewise, other software using the artifact may fail to integrate. These violations have the potential to cause failure, thus making the change a BREAKING CHANGE.

The expectations could be embodied as contracts or assumptions.

The contracts subject to change when a new artifact replaces an old one could be:

  • Syntactic. For example, the number of arguments required by a method call changes from 2 to 3.
  • Semantic. For example, the amount returned by a service changes from comprising no taxes to including the sales tax.
  • Behavioral. For example, a function call changes from being side-effect free to having side-effects.
  • Quality of service (QoS). For example, a credit check service’s availability changes from 6AM-6PM to 2AM-midnight.

Assumptions can be violated. Garlan [ref] discusses the consequences of dealing with implicit assumptions in the context of composite applications.

BREAKING CHANGEs may keep the users of your software artifact from replacing the old with the new. This impacts maintainability in a negative manner and thus may not be feasible.

Substitutability Check

People use software that is subject to change. Since their correct operation depends on this software they want to assess whether a software update (i.e., replacing an old piece of software such as a library with a new one) would cause their system to break.

Can I substitute the old with the new?

Answering this question entails checking the effect of replacing the old with the new.

The easiest way is to replace the old with the new and see whether the system still works as it should. However if there are problems doing so may have irreversible consequences (e.g., life support system). Even if you can do it, what guarantees do you have that your observations cover all the possible effects of the changes? You need a means of testing for breaking changes that doesn’t impact the existing system/has low risk, and offers great accuracy (i.e., no misses).

Therefore:

Perform a substitutability check to determine whether a software artifact (library, component, service, etc.) can be replaced with a new one without causing a breaking change.

No side-effects translates into performing the check outside the live environment.

There are 2 options, analytical and empirical. High accuracy cannot be achieved in through an empirical check unless the artifact is small and simple. An analytical check requires iterating through all changes and assessing their potential impact. This is impossible without a list of what has changed (i.e., a diff), understanding the changes and their potential interactions and side effects. This is feasible only if you have visibility inside the artifact and understand how it works.

There are several dimensions to performing a SUBSTITUTABILITY CHECK.

One dimension covers who performs the check:

  • Person (manual)
    • Read the changes file and assess impact
    • Perform visual inspection
    • Try it out
    • Perform formal analysis
    • Run test suite
  • Software (automatic)
    • Manager component

Another dimension covers when is the SUBSTITUTABILITY CHECK being performed:

  • At evaluation time. You receive an update and decide whether to substitute (upgrade)
  • At build time. The IDE/compiler must pick between the old and the new library to link against
  • At run time. The runtime must pick between the old and the new dynamic library to load. As with other late binding mechanisms, going this route requires accounting for a failed check at run time.

The SUBSTITUTABILITY CHECK is a procedure that determines whether replacing an old software artifact with a new one will cause BREAKING CHANGEs. The next pattern answers the same question without requiring complex analysis.

[More coming soon so stay tuned]

Process Models for the Masses

A decade ago few people dealing with code were interested in business processes. That’s no longer the case. Due to a variety of reasons many developers discuss processes in one form or another. Some want to understand the business process that the appplication they’re building implements. Others want to grok the coordination of the services comprising their SOA. Yet others leverage workflow engines to implement the flow of pages in web applications (i.e., page flow).

A large fraction of these folks assume that processes always involve activities and their sequencing. Right? Wrong! Just because the workflow/orchetsration/coordination/etc. engine they picked to implement your process revolves around activities doesn’t mean that that’s the only way to think about a process. There are many instances when representing a process as a set of activities is unfeasible. Consider for example a web-based shoppping site: at any point you can perform one out of many activities: add something to your shopping cart; remove something from your cart; checkout; update your profile; save the cart for another visit; and so on. How would you represent somethig like this with activities?

What’s missing is the process model, something that not many newcomers are aware of. In a nutshell the process model provides the set of abstractions, relationships, and constraints that allows people to define processes. Several process models are available, each of which is suitable for a particular class of problems.

The most popular process model is activity-based. Its abstractions represent the activities performed as part of the process (or the states between these activities). This model is suitable when the process can be described through activities. It is also easy to understand because the activities resemble the elements of structural programming. However, as the web-based shopping example illustrates, sometimes it’s cumbersome to express a process as a set of activites.

Another process model uses conversations among participants as the fundamental abstraction. This model has its roots in social sciences and was pioneered by Terry Winograd (a computer scientist) and Fernando Flores (a philosopher). Together they founded Action Technologies, a company that in the mid-1980s built the first workflow system employing a conversation-based process model). Winograd and Flores describe the ideas behind their Business Interaction Model and ActionWorks in Understanding Computers and Cognition. Anyone serious about workflow ought to read their book.

A third process model uses the artifact produced as the process unfolds as its key abstraction. Though not as popular as the activity-based process model the artifact is well suited for representing processes where at any point one out of many actions are possible, such as an e-commerce Web site. The work of Richard Hull and Jianwen Su (among others) focuses on this process model. AT&T’s Vortex workflow system represents a research prototype built around the artifact-based process model.

I’m sure there are other process models, including hybrid ones that combine some of the above. Unfortunately the literature is scarce in this area. In addition, the increasing popularity of the activity-based process model (mostly because people don’t know any better) will only bury the other models even deeper.