Category Archives: work

Live Labs Listas

Right before leaving for OOPSLA 2007 I noticed that my colleagues from Live Labs released Live Labs Listas. Listas is a great example of the user experience that can be achieved in modern web applications (i.e., Web 2.0). For that reason a couple of months ago I invited a member from the Listas team to showcase the UX features such as responsiveness, drag-and-drop, etc. to p&p’s Web Client Software Factory team. Great to see it ship and I look forward to other sites providing a user experience on a par with Listas!

Patterns Article in IEEE Software

My article The Growing Divide in the Patterns World (co-authored with Wojtek Kozaczynski, Ade Miller and Jason Hogg) appeared in the Software Patterns issue of IEEE Software (July/August 2007).

Here’s the abstract:

Recently Microsoft’s patterns & practices group conducted a survey that indicates a significant gap between the patterns expert community and the software practitioners attempting to use patterns and leverage pattern thinking in their daily work. As this gap widens it will lead to an irreversible divide between the two communities, and patterns will fail to deliver their potential. In the light of our experience using patterns to package development guidance as well as input from practitioners using patterns we analyze the key causes behind this gap and recommend a set of actions aimed at the patterns community. Bridging this gap will have a dramatic effect on practitioners’ understanding of patterns and their ability to leverage them as the patterns expert community envisions.

More info off their web site (assuming the server is responding, which is not the case now).I’m glad to see that after over 12 years software patterns are still interesting. How many of the hot topics of 1994 are still talked about today?

Don’t Miss this QCon Keynote: Democratizing The Cloud

If you’re going to next week’s QCon in London do yourself a favor and attend Erik Meijer’s keynote Democratizing The Cloud. Here’s Erik’s abstract:

The web is rocking the world of developers. Our customers love consistency. They want to have the same rich experience, anywhere, any time, on any device. Our sales people love market share. They want no platform that cannot leverage their web services. We ourselves have embraced agile methods. We want to keep our options open as long as possible and create software incrementally by successive refactorings. This surely sounds like a contradiction, another impossible triangle. As the Dutch artist MC Escher once said “Only those who attempt the absurd will achieve the impossible”. Hence we are trying to stretch the .NET framework to cover the Cloud such that it will become possible to incrementally and seamlessly design, develop, and debug complex distributed applications using your favorite existing and unmodified .NET compiler and deploy these applications anywhere.

Erik will show what consistency means; he will demonstrate how agilty translates into what you do right after you go to the File menu; he will prove how you can keep your options open until just about the very end. So if you’re only planning on attending a single QCon keynote, I strongly recommend Democratizing The Cloud. A few hours ago Erik, Jeff Van Gogh and myself reviewed the demos that he’ll show next week in London. You won’t believe me if I’d tell you, so better see for yourself. Do trust me though that they’re cool–I did work on the one involving validation. Hint: you’ll see p&p’s Enterprise LIbrary like you’ve never seen it before.

The Real Inversion of Control

A recent conversation with my friend and colleague Erik Meijer provided the nudge to finally write about something has been bothering me for some time. So here it goes…The popular view about Inversion of Control (IOC) is that it represents a technique for removing dependencies in object-oriented programs. People immediately think of their favorite dependency injection framework, whatever that may be. That’s nice and well, but is that the real IOC?Consider the following:

  • XML was pitched as a language specifically designed to make parsing easy so developers would not have to write another parser. That has happened allright, very few worry about parsing XML today. However while computers spend less cycles parsing well-formed markup, people crippled their languages with angle brackets, end tags, and a verbose syntax. This in spite the fact that the number of language users > number of XML parsers.
  • Good SOA design recommends building stateless services. Service implementations that follow these guidelines retain nothing from one invocation to another. However while the services do almost nothing to maintain the state of the service conversation, the developers building service consumers picked up the tab. Again, the number of service consumers > the number of services.
  • Web browsers allow people to interact with applications through widgets such as text boxes, radio buttons, check boxes, buttons, and a few more. These widgets are so primitive that they easily map into the native widgets provided by the virtually all contemporary host environments. However while the browser happily renders widgets with consipcuous text UI era traits, developers jumped through many hoops to push the user experience closer to what the platforms have been providing for years. And you guessed it, the number of web applications > the number of browsers.

There are other examples but I’ll stop at 3. The commonality across them is that in each case the software is simpler somewhere because many developers picked up the tab somewhere else. Instead of localizing complexity and putting the machines to work so we could enjoy other, more interesting challenges, the opposite happened. We now have to write in a verbose, ugly-looking language that the software can easily process; manage the state of service conversations so the servers would not bother about it; use plugins and pull lots of strings to give the user experience a face lift so the browsers do minimal processing and rendering; and so on. So consider this other way to look at IOC: the software is in control, and we’re working for it. Darn did I take the blue pill?

Just shipped: WSSF and WCSF

Two of the projects I’ve been involved with since I started in Microsoft’s p&p group have just shipped:

Both ship in source code and include reference implementations.

If you haven’t heard of software factories my colleagues Jack Greenfield, Keith Short, Steve Cook, Stuart Kent have an excellent book with the same title.

Software Patterns Survey

In the patterns & practices (p&p) group at Microsoft we have been using software patterns for several years. I’m working with a couple of colleagues on gauging how the folks employing the guidance coming out of p&p (such as guides, application blocks, software factories and reference implementations) have, are, and will be using software patterns.

To do that we assembled a short survey. If you’ve used patterns I encourage you to take it. If you haven’t used patterns yet but your development tools use patterns one way or another then you could also take it since I’m also interested in your perspective.

You can find the survey Software Patterns: Past, Present, Future here. Thank you in advance for your answers.

Pattern Languages of Program Design

After a long gestation the fifth volume of Pattern Languages of Program Design (PLoPD) has been published.

Myself, Markus and James selected among patterns workshopped at PLoP conferences from 1998 through 2004. We structured the book in six parts. Part I focuses on design and contains patterns aimed at people designing object systems. As the Internet and embedded systems continue to expand their reach they bring with them concurrency and resource management problems; Part II contains patterns on these topics. Part III continues the shift from one to many applications and contains patterns for distributed systems. The domain specific patterns from Part IV focus on mobile telephony and Web-based applications. Part V shifts gears to architecture and comprises patterns that tackle composition, extensibility, and reuse. Finally, Part VI offers a smorgasbord of meta-patterns for improving the quality of pattern papers and helping their authors.

Here’s what you can find in each chapter:

  1. The Dynamic Object Model pattern combines ideas from class-based (like Java) and prototype-based (like Self) languages to address dealing with elaborate flexibility requirements.
  2. The Domain Object Manager allows application code to handle transient and persistent domain objects while supporting multiple data stores or application servers, keeping the domain objects independent of the persistence or middleware APIs.
  3. Encapsulate Context provides value to developers who are seeking ways to lower the ripple effects of code changes regardless of programming language, allowing them to manage an increasing number of call parameters without introducing global variables.
  4. A Pattern Language for Efficient, Predictable, Scalable, and Flexible Dispatching Components addresses the challenges associated with developing dispatching components, providing one of the building blocks of a handbook for distributed real-time and embedded middleware.
  5. Triple-T focuses on writing software for real-time processing is hard and poses unique challenges, covering five patterns harvested from time-triggered bus architectures for safety-critical real-time systems, such as the ones employed by Airbus airplanes or BMW and DaimlerChrysler automobiles.
  6. Real Time and Resource Overload Language presents a pattern language for designing reactive systems that gracefully accommodate load bursts, applicable to any systems that process incoming requests, such as web servers, middleware, OLTP, and so on.
  7. Drawing upon examples from several systems that deal with distribution De-Centralized Locking covers a pattern for managing locks in the context of distributed systems.
  8. The Comparand Pattern focuses on dealing with identity when working with objects from different hosts or processes within a distributed system.
  9. Service Discovery tackles a common problem in practical ways, distilling solutions from well-established examples such as SLP, JXTA, UPnP, LDAP, DNS, and the now ubiquitous and IEEE 802.11.
  10. MoRaR is a pattern language focused on mobility and radio resource management.
  11. Content Conversion and Generation on the Web is a pattern language aimed at people building applications that deal with dynamic HTML generation.
  12. Plug-ins represent a popular technique for extending applications, allowing users to late-bind new functionality. The Patterns for Plug-Ins pattern language covers techniques mined from a wide body of software, including operating systems, web browsers, graphics programs, and development environments.
  13. The Grid Middleware Architectural Pattern covers the architectural elements of grid middleware as well as guidelines for implementing and deploying them.
  14. Targeting application developers integrating components written in different languages or built with hetereogenous component concepts, the Patterns of Component and Language Integration distill insight from systems that include Apache Axis and the Simplified Wrapper Interface Generator (SWIG).
  15. Patterns for Successful Framework Development
    Frameworks
    covers a set of patterns for mitigating the mismatch between the recommended practice of building frameworks and the reality of many software development projects.
  16. Distilling from 10 years of writing and reviewing patterns Advanced Pattern Writing provides advice for improving pattern writing.
  17. A Language Designer’s Pattern Language explains how does one generate a pattern language from a system of forces.
  18. The Language of Shepherding focuses on analyzing and providing feedback about patterns.
  19. Patterns of the Prairie Houses uses the design themes of Frank Lloyd Wright’s prairie houses as an exploratory vehicle for showing how people should approach pattern mining and writing.

10 Years After: Developments and Trends in the Patterns World

I gave this talk in the Patterns – Past, present, future track at JAOO, in Aarhus, Denmark.


Most newcomers think that patterns are limited to the topics that made a big splash about 10 years ago. In fact many other patterns and pattern languages are quietly put to work, helping
practitioners solve problems in a growing number of areas such as enterprise architectures, AOP, middleware and grid computing, embedded systems, performance tuning, and so on. In this session I tour some of the most interesting developments and trends in the patterns world based on my experience with editing the 5th volume of the Pattern Languages of Programs book series. This overview will give you insight into current patterns work, prepare you for finding patterns for the problems that you’re interested in, as well as help you position in context patterns that you may consider authoring.

You can download the PDF from my talk list.

JAOO 2005 Panel on the Future of Patterns

This year I am speaking at JAOO. On Monday, September 25 I’ll be participating in a panel on the future of patterns with Markus Völter, Frank Buschmann, Kevlin Henney, and Klaus Marquardt. I know everybody on the panel and thus am looking forward to a lively and interesting discussion. If you’d be interested in participating but cannot make it please post questions as a comment below; I’ll add them to the questions from the audience. After the panel I will try to summarize the discussion here so stay tuned.

Framework Abuse

The Problem

Most IT projects involve frameworks. End users like to talk about frameworks when they discuss new applications. Managers add frameworks to their project plans. Architects design frameworks. Developers build frameworks. Currently many IT projects involve framework development. To an external observer or to someone relatively new to programming frameworks appear as a required ingredient of modern software projects. Is this true? Should projects invariably embark on developing frameworks? What should people be aware of when planning, designing, building, or using frameworks? Many frameworks waste time and money. How do you know that your framework project won’t?

Background

Throughout the last 18 months I worked on several IT projects at large financial services companies. I noticed that almost invariably building new software systems includes the development of one or several frameworks. Examples include (in no particular order) testing framework; application integration framework; common services framework; messaging framework; application architecture framework; connection framework; reference data framework; persistence framework; logging framework; caching framework; authorization framework; portal framework; rules management framework; optimization framework; business process management framework; data management framework; presentation layer framework; and so on. In addition, almost all project stakeholders use the term “framework.”

In fact over 80% of these projects had actually abused frameworks. In other words: the framework did not deliver the promises: increased reuse, faster development, better flexibility. These observations don’t fit with my understanding of frameworks’ purpose, how they begin, who builds them, and how they evolve.

What Are Frameworks?

Frameworks promise (and properly crafted also deliver) design and code reuse. However, reuse has prerequisites.

Crafting reusable code requires generalization. Without several like-systems to generalize from, creating reusable designs and writing reusable code remains wishful thinking [Roberts and Johnson].

Reuse also has costs. It costs 1.5 to 3 times as much to create and support a reusable component as it does to implement a similar component for a single application. Likewise, it takes 2-3 product cycles for the benefits of reuse to become significant [Jacobson].

Symptoms

Unfortunately many projects embarking on framework development do not seem to look at the reuse and cost considerations associated with frameworks. In contrast, several common traits of these projects point out a mismatch between the existing body of work and the current use of the term “framework.” This mismatch causes miscommunication between those who use the term appropriately and those who abuse it (regrettably in growing numbers). The mismatch also has implications on the cost, expectations about, and success of software development projects involving frameworks.

Some of the traits that identify this mismatch follow.

  • Building frameworks without considering reuse Generally people have some idea about some connection between frameworks and reuse. Consequently many frameworks tend to appear in areas that could benefit from reuse. However, few projects actually reused frameworks. Most frequently the framework serves a single application. Sometimes projects even abandon the framework under development and start building a new one.
  • Building frameworks without considering the cost of reusable
    software
    Although not a TLA reuse is a buzzword. Most project managers know that associating their projects with reuse makes them more likely to survive the scrutiny of upper management. Who cares about the ROI? However, the same managers waving the reuse flag do not bring up its cost implications. The promise of cost savings through reuse looks good on proposals. The certitude that the initial costs are going to be 1.5-3 times higher typically triggers an ROI analysis.
  • Building frameworks without concrete applications Many projects include frameworks when no applications to generalize from are available. Even if there are, they’re likely to use different, arcane technologies. The people eager to build frameworks would rather tinker with the newest technologies rather than spend their time harvesting common abstractions from fossils. They end up chasing and struggling with understanding the requirements. They focus on defining and solving the problem rather than generalizing from existing solutions.
  • Building frameworks and the first applications using them concurrently Many people end up building frameworks at the same time with the first applications that use them. This seems to make sense since the same developers are working on both. However, it places framework development on the critical path. The generalization work has not other choice but proceed within the time constraints of the application. In addition, should the team need to cut corners, framework development is the first candidate.
  • Building frameworks and assume that they’re not going to change
    Project plans that include framework building typically show that their development stops before (or in the best case at the same time with) the work on the application for which they’re built completes. This would seem reasonable if the framework is merely a piece of scaffolding necessary to achieve a one-time goal. However, setting aside the issue of putting framework development on the critical path, building frameworks does not end when the last line of version 1.0 is written [Roberts and Johnson]. Like good wine, frameworks need to mature. Actual use in several applications provides the builders with valuable insight into the parts that need to be flexible and the ones that do not, closing the feedback loop. Refactoring around the emerging abstractions is a requirement for the evolutionary process. Neglecting it is likely to yield brittle constructs that developers cannot reused outside the precise context in which they were designed.
  • Building frameworks without the required know-how Sometimes projects are so enamored with frameworks that they embark on building them regardless of whether they have the know-how. “Really smart people working on it” so the effort is in good hands. However, intelligence is not a substitute for experience. As skeleton applications frameworks require experience with the application domain. Developers end up learning about the domain while building the framework. They focus on acquiring the know-how rather than identifying and generalizing the reusable parts.
  • Building frameworks to get buy-in from the business Many projects tend to be tech-driven rather than business driven. Rather than the business users seeking a solution to a business problem, the tech side already has a solution (typically a new technology or a tool) and is looking for problems to solve with it. They talk the business users (who ultimately fund projects) into giving the green light by telling them that they’re building frameworks. The business users don’t really know the hows, whats and whens of frameworks. Consequently they go ahead and give the go-ahead in spite of not knowing what they’re getting. Naturally this works only if the business people don’t know anything about frameworks and, like with the emperor’s new clothes, they chose to sweep their ignorance under the rug.

Do you have additional thoughts or suggestions for solving this problem? Add your own comments below.