Tag Archives: patterns

Patterns Article in IEEE Software

My article The Growing Divide in the Patterns World (co-authored with Wojtek Kozaczynski, Ade Miller and Jason Hogg) appeared in the Software Patterns issue of IEEE Software (July/August 2007).

Here’s the abstract:

Recently Microsoft’s patterns & practices group conducted a survey that indicates a significant gap between the patterns expert community and the software practitioners attempting to use patterns and leverage pattern thinking in their daily work. As this gap widens it will lead to an irreversible divide between the two communities, and patterns will fail to deliver their potential. In the light of our experience using patterns to package development guidance as well as input from practitioners using patterns we analyze the key causes behind this gap and recommend a set of actions aimed at the patterns community. Bridging this gap will have a dramatic effect on practitioners’ understanding of patterns and their ability to leverage them as the patterns expert community envisions.

More info off their web site (assuming the server is responding, which is not the case now).I’m glad to see that after over 12 years software patterns are still interesting. How many of the hot topics of 1994 are still talked about today?

Patterns for Fault Tolerant Software

My fellow Hillsider Robert S. Hanmer has just finished his book on fault tolerant patterns (to be available from John Wiley & Sons, October-November 2007). Here’s the book’s descritiption, in Bob’s words:

This book presents proven techniques to achieve highly available, fault tolerant software that can be implemented by software developers, software architects and small teams. The techniques are presented in the form of patterns as a resource for teaching developers and students about fault tolerance principles and also as a reference for experts seeking to select the technique appropriate for a given system.

Within the phases of fault tolerance (fault detection, error processing and fault treatment) the patterns will be organized in a way that leads from high-level abstractions to the concrete mechanisms. The collection of techniques will be programming language independent, and will be presented in a way that supports their working together to design fault tolerant software. This allows the designer to build the fault tolerant pattern language needed to solve their unique design problems.

Readers are guided from concepts and terminology, through common principles and methods to advanced techniques and practices in the development of software systems.

This book gives present proven methods of increasing the fault tolerance in a way that helps individual architects and developers. The fault tolerance domain lacks a handbook that provides well-known techniques and practices that are larger in scope than ‘defensive programming’ as many books focus on specific techniques or exclusively on reliability engineering.

I’ve been following Bob’s work and I am very happy to see it come to fruition in this form. For years seasoned programmers have been extracting techniques and insight from carrier grade software. It’s great to see a book on this topic from one of the long time members of the patterns community!

Software Patterns Exhibit Positive Network Effects

I’ve just wrapped up work on a patterns article. While doing the analysis with my co-authors (an co-workers) Ade Miller and Wojtek Kozaczynski I wondered about the connection between software patterns and Metcalfe’s Law.

By the way, if you’re interested in reading about how Metcalfe came up with the ideas that ultimately led to Ethernet read Michael Hiltzik’s book; you will thank me later.

Clearly the more people know about patterns, the better you can communicate with them (since instead of waving hands you would say Decorator, Broker or Service-Oriented Decomposition). Regardless of whether the growth is quadratic or not, the value of patterns increases with the number of people who use them. Stated otherwise, patterns exhibit positive network effects.

Software Patterns Survey

In the patterns & practices (p&p) group at Microsoft we have been using software patterns for several years. I’m working with a couple of colleagues on gauging how the folks employing the guidance coming out of p&p (such as guides, application blocks, software factories and reference implementations) have, are, and will be using software patterns.

To do that we assembled a short survey. If you’ve used patterns I encourage you to take it. If you haven’t used patterns yet but your development tools use patterns one way or another then you could also take it since I’m also interested in your perspective.

You can find the survey Software Patterns: Past, Present, Future here. Thank you in advance for your answers.

A Pattern Language for Versioning

Global Context: Software Change

You are releasing a piece of software (i.e., a software artifact) for others to (re)use as a building block in their software. Reusable code fragments could be delivered as libraries, software components, applications, services, etc. Others find your artifact useful and begin using it. They could use the functions or classes within your library; they assemble applications out of your component(s); they compose your application in their workflows; they build business processes that invoke your service(s).

Once others start reusing your software you discover that you need to modify it. There are instances where modification is not possible. For example, software controlling deep sea sensors or satellite equipment may be out of reach and thus impossible to change, even when its users would like to do so. However nowadays that is the exception rather than the norm. But why change software artifacts that others are using?

One common reason for software change is a bug fix. In spite of stringent testing procedures the probability of finding errors after the software passes quality assurance and ships is not zero. Consequently you (or those using your software) may discover an error in released software. After understanding what is causing the error you modify the code to correct it.

Another driver for change is evolution. The requirements change. For example, new legislation may mandate compliance with a standard. Alternatively there could be a change in the environment. For instance, one of the systems your software is integrating with is being phased out and replaced with a new one. The new systems have slightly different interfaces and your software must change to accommodate them.

Regardless of reason changing software could cause failures—an undesired effect—even when the change fixes an existing error. Consequently the value of the change is context-dependent. For some users the change critical: they cannot operate without it. For others the cost of accommodating the change may be higher than the cost of compensating for it: their software may be counting on the bugs being there, and removing them would entail significant work.

Software change and its impact on those using the software provide the shared context for this pattern language.

Non-breaking Change

Not all types of software change cause failure, having an undesired effect on its users. Sometimes nothing breaks following a change. Ideally all changes would non-breaking. In reality this is the exception rather than the rule, which makes this case uninteresting.

Breaking Change

You released a software artifact. After the release you change it to accommodate a bug fix or a new requirement. You release a new artifact that replaces the old one. However its users start having problems; clearly the shift from the old artifact to the new one is not a NON-BREAKING CHANGE.

Why would a software change cause failure?

A change breaks software when the new artifact (i.e., the software with the change applied) violates one explicit or implicit expectations that the users of, or other software that depends on it, have about the old artifact.

Therefore:

A change that violates an expectation about a software artifact impacts its dependents. User interacting with the artifact may experience unexpected behavior. Likewise, other software using the artifact may fail to integrate. These violations have the potential to cause failure, thus making the change a BREAKING CHANGE.

The expectations could be embodied as contracts or assumptions.

The contracts subject to change when a new artifact replaces an old one could be:

  • Syntactic. For example, the number of arguments required by a method call changes from 2 to 3.
  • Semantic. For example, the amount returned by a service changes from comprising no taxes to including the sales tax.
  • Behavioral. For example, a function call changes from being side-effect free to having side-effects.
  • Quality of service (QoS). For example, a credit check service’s availability changes from 6AM-6PM to 2AM-midnight.

Assumptions can be violated. Garlan [ref] discusses the consequences of dealing with implicit assumptions in the context of composite applications.

BREAKING CHANGEs may keep the users of your software artifact from replacing the old with the new. This impacts maintainability in a negative manner and thus may not be feasible.

Substitutability Check

People use software that is subject to change. Since their correct operation depends on this software they want to assess whether a software update (i.e., replacing an old piece of software such as a library with a new one) would cause their system to break.

Can I substitute the old with the new?

Answering this question entails checking the effect of replacing the old with the new.

The easiest way is to replace the old with the new and see whether the system still works as it should. However if there are problems doing so may have irreversible consequences (e.g., life support system). Even if you can do it, what guarantees do you have that your observations cover all the possible effects of the changes? You need a means of testing for breaking changes that doesn’t impact the existing system/has low risk, and offers great accuracy (i.e., no misses).

Therefore:

Perform a substitutability check to determine whether a software artifact (library, component, service, etc.) can be replaced with a new one without causing a breaking change.

No side-effects translates into performing the check outside the live environment.

There are 2 options, analytical and empirical. High accuracy cannot be achieved in through an empirical check unless the artifact is small and simple. An analytical check requires iterating through all changes and assessing their potential impact. This is impossible without a list of what has changed (i.e., a diff), understanding the changes and their potential interactions and side effects. This is feasible only if you have visibility inside the artifact and understand how it works.

There are several dimensions to performing a SUBSTITUTABILITY CHECK.

One dimension covers who performs the check:

  • Person (manual)
    • Read the changes file and assess impact
    • Perform visual inspection
    • Try it out
    • Perform formal analysis
    • Run test suite
  • Software (automatic)
    • Manager component

Another dimension covers when is the SUBSTITUTABILITY CHECK being performed:

  • At evaluation time. You receive an update and decide whether to substitute (upgrade)
  • At build time. The IDE/compiler must pick between the old and the new library to link against
  • At run time. The runtime must pick between the old and the new dynamic library to load. As with other late binding mechanisms, going this route requires accounting for a failed check at run time.

The SUBSTITUTABILITY CHECK is a procedure that determines whether replacing an old software artifact with a new one will cause BREAKING CHANGEs. The next pattern answers the same question without requiring complex analysis.

[More coming soon so stay tuned]

Pattern Languages of Program Design

After a long gestation the fifth volume of Pattern Languages of Program Design (PLoPD) has been published.

Myself, Markus and James selected among patterns workshopped at PLoP conferences from 1998 through 2004. We structured the book in six parts. Part I focuses on design and contains patterns aimed at people designing object systems. As the Internet and embedded systems continue to expand their reach they bring with them concurrency and resource management problems; Part II contains patterns on these topics. Part III continues the shift from one to many applications and contains patterns for distributed systems. The domain specific patterns from Part IV focus on mobile telephony and Web-based applications. Part V shifts gears to architecture and comprises patterns that tackle composition, extensibility, and reuse. Finally, Part VI offers a smorgasbord of meta-patterns for improving the quality of pattern papers and helping their authors.

Here’s what you can find in each chapter:

  1. The Dynamic Object Model pattern combines ideas from class-based (like Java) and prototype-based (like Self) languages to address dealing with elaborate flexibility requirements.
  2. The Domain Object Manager allows application code to handle transient and persistent domain objects while supporting multiple data stores or application servers, keeping the domain objects independent of the persistence or middleware APIs.
  3. Encapsulate Context provides value to developers who are seeking ways to lower the ripple effects of code changes regardless of programming language, allowing them to manage an increasing number of call parameters without introducing global variables.
  4. A Pattern Language for Efficient, Predictable, Scalable, and Flexible Dispatching Components addresses the challenges associated with developing dispatching components, providing one of the building blocks of a handbook for distributed real-time and embedded middleware.
  5. Triple-T focuses on writing software for real-time processing is hard and poses unique challenges, covering five patterns harvested from time-triggered bus architectures for safety-critical real-time systems, such as the ones employed by Airbus airplanes or BMW and DaimlerChrysler automobiles.
  6. Real Time and Resource Overload Language presents a pattern language for designing reactive systems that gracefully accommodate load bursts, applicable to any systems that process incoming requests, such as web servers, middleware, OLTP, and so on.
  7. Drawing upon examples from several systems that deal with distribution De-Centralized Locking covers a pattern for managing locks in the context of distributed systems.
  8. The Comparand Pattern focuses on dealing with identity when working with objects from different hosts or processes within a distributed system.
  9. Service Discovery tackles a common problem in practical ways, distilling solutions from well-established examples such as SLP, JXTA, UPnP, LDAP, DNS, and the now ubiquitous and IEEE 802.11.
  10. MoRaR is a pattern language focused on mobility and radio resource management.
  11. Content Conversion and Generation on the Web is a pattern language aimed at people building applications that deal with dynamic HTML generation.
  12. Plug-ins represent a popular technique for extending applications, allowing users to late-bind new functionality. The Patterns for Plug-Ins pattern language covers techniques mined from a wide body of software, including operating systems, web browsers, graphics programs, and development environments.
  13. The Grid Middleware Architectural Pattern covers the architectural elements of grid middleware as well as guidelines for implementing and deploying them.
  14. Targeting application developers integrating components written in different languages or built with hetereogenous component concepts, the Patterns of Component and Language Integration distill insight from systems that include Apache Axis and the Simplified Wrapper Interface Generator (SWIG).
  15. Patterns for Successful Framework Development
    Frameworks
    covers a set of patterns for mitigating the mismatch between the recommended practice of building frameworks and the reality of many software development projects.
  16. Distilling from 10 years of writing and reviewing patterns Advanced Pattern Writing provides advice for improving pattern writing.
  17. A Language Designer’s Pattern Language explains how does one generate a pattern language from a system of forces.
  18. The Language of Shepherding focuses on analyzing and providing feedback about patterns.
  19. Patterns of the Prairie Houses uses the design themes of Frank Lloyd Wright’s prairie houses as an exploratory vehicle for showing how people should approach pattern mining and writing.

John Vlissides

John Vlissides, one of the Gang of Four, passed away at the end of November. John had made significant contributions to the software patterns community as an author, series editor, speaker, and so on.

I first met John at OOPSLA 1998 in Denver. I already knew a few things about his involvement in the GoF book but nothing compares to meeting someone in person. During one of the OOPSLA receptions we talked for about one hour. The conversation started with my telling John about my research on lightweight workflow architectures at the University of Illinois. We coevered much more though, well beyond patterns. I remember John’s advice about the pursuit of an academic career, and his take on the childcare system in the US.

Following 1998 I ran into John at OOPSLA conferences, and sometimes at the annual post-OOPSLA Hillside event. In 2003 in Annaheim, CA John asked me whether I’d be interested in editing the PLoPD5 volume in his Patterns Series. I don’t remember his exact words, but he said something along the lines of his being impressed with my completing and defending a dissertation in time.

I last saw John at OOPSLA 2004 in Vancouver, BC. Although he was very busy running the conference we had a good chat.

Besides his family John will also be missed by his patterns friends.

Listening to Hillside fellows in Tampa, FL (2001)
jv-hillside-2001-2.jpg

Drumming at the Experience the Music Project in Seattle, WA (2002)
jv-drums-2002.jpg

Talking to Hillside fellows in Tampa, FL (2001)
jv-hillside-2001.jpg

JAOO 2005 Panel on the Future of Patterns

This year I am speaking at JAOO. On Monday, September 25 I’ll be participating in a panel on the future of patterns with Markus Völter, Frank Buschmann, Kevlin Henney, and Klaus Marquardt. I know everybody on the panel and thus am looking forward to a lively and interesting discussion. If you’d be interested in participating but cannot make it please post questions as a comment below; I’ll add them to the questions from the audience. After the panel I will try to summarize the discussion here so stay tuned.

Patterns Projects in Progress

I assembled a list of current pattern projects for my JAOO talk 10 Years After: Developments and Trends in the Patterns World (available in PDF form from my talk list). Ralph Johnson encouraged me to publish this list so that interested folks could consult it, and more importantly, update it. Here it is; feel free to add new pointers or comment on the existing ones.

Books in Progress

Other Projects