Tag Archives: Design

Essential Software Design Documents

When it comes to documenting software designs, there are many practices ranging from a quick whiteboard sketch to fully specified UML models with OCL to tens of pages of written text. Here is what I found that worked best based on most projects that I have been part of:

  1. Static Structure Diagram: this is a class, module, or package diagram. This diagram (though often it is more than one), shows the relationship between parts of the system at a given level of abstraction. The relationships shown include: containment, references, parameters, inheritance, and so on. Record on the diagram indirect and implied dependencies with a note.
  2. Initialization Sequence Diagram: shows how the system is brought up from being on the disk into the operative memory and “ready to transact business” state. This diagram is part of the dynamic model of the system. It shows the order of messages sent or method invocations. Often also shows resources allocated to get up and running.
  3. Shutdown Sequence Diagram: shows how to put back the system or component into the proverbial “bit bucket.” Must complement the initialization sequence diagram and demonstrate that the system is not loosing resources during its operation.
  4. Sequence Diagrams, Activity, and/or State Diagrams for the key “business” of the system or software part. All other activities took place just to get ready to do the business of the system. Now it is time to get the business done. The key sequence diagrams show what the system does, how it provides the value that it is expected to deliver.
  5. Brief Design Commentary to explain the “why” behind key structural and behavioral elements and decisions. This is usually one Wiki page that ties all other documents together into a coherent whole.

Each of these can be created at “every zoom level” in the system. This means that if you are developing a component-based system, then you would have this set of design documents for the system, subsystems, modules, and components. Within each you represent the unique concerns of each software part.

Read Applying UML and Patterns, 3rd. Ed., by Craig Larman for an in-depth course on how to do object-oriented analysis and design on an iterative software development project. As you do design more, review your results, learn from your mistakes, and over time you’ll get better at it.

Resources:

  1. UML basics: An introduction to the Unified Modeling Language, from IBM developerWorks, 2003.
  2. Allen Holub’s UML Quick Reference. 2007.
  3. Book excerpt from OOAD with Applications, by Grady Booch.
  4. UML Modeling in Color, from Wikipedia.

More on Prototyping

To convert the unknowns into knowns, you need to aggressively prototype.

What to Prototype?

Here are some examples:

Prototype the use of an API. Create a prototype to see if the API that you need to use works the way you think it does. This includes confirming that it can be called with parameters of the type and range that you can provide. The API may belong to a component that one of your team mates wrote, or it may belong to a third party library. Either way, you want to be absolutely certain about how to use it. No amount of documentation reading will substitute for a well pointed prototype.

Prototype an algorithm. Create a prototype to prove to yourself beyond reasonable doubt that you can implement the solution to a problem.

Prototype an interaction. Create a prototype to show that the interaction sequence or method that you want to use in your application actually works for the intended use. You can take the prototype and usability test it, incorporate the feedback, and then place it into your application.

What should the prototype look like?

Since we are talking about software development, your prototype should be a software program. A small program. As small as possible. A program that answers one or two critical questions and that’s it.

The prototype should be like the small program that you write to prove to the API writers that their API has a defect. It has to be small and prove one point. Your objective from the prototype to learn something that you didn’t know, or confirm something that you suspected.

How to Prototype?

Pick one or two key questions that you want your prototype to answer and focus your small program answer them. Keep in mind common software engineering and experimentation principles as you decide on how to proceed:

  1. divide & conquer: write multiple prototypes as needed
  2. reduce scope: keep the scope of each prototype to a minimum
  3. isolate concerns: separate the concerns so that you can establish root cause
  4. confirm behavior: confirm that an API behaves exactly the way you had thought that it would under the conditions that you anticipate
  5. validate assumptions: validate that what you think is true, it is indeed so
  6. validate invariants, pre- and post-conditions: half the battle is to identify these; you get extra credit for validating them

I have noticed that after a while some prototypes that started out with the best of intentions, become monsters. Don’t be tempted to stick another one or two topics into an existing prototype. Your prototype will become so complicated that you cannot confirm root causes for outcomes. Then you lost your ability to know things for sure.

Lastly: the most important rule is a well worn one: KISS=Keep It Simple & Sufficient.

Why Prototype?

Every once in a while I am surprised by somebody who says that prototyping is such a waste of time. For quite some time I have taken for granted that prototyping is a fact of my software engineering life. Maybe it is worth asking: what is the problem for which prototyping is the answer? Or, in other words: why should you prototype?

You should prototype when you don’t know enough about about the problem. Which is almost always. When your knowledge is lacking in some aspect, then it is time to write a prototype. When you don’t know enough about an API, some requirement, the operating system that you are designing for, the libraries that you want to use, and so on. When you feel that you have to answer a few clarifying questions, then it is time to answer them with a prototype.

The short answer is that you prototype because you don’t know enough to proceed with the project at an acceptable level of risk.

Common Defects

A while back I had the opportunity to analyze a few defects up close. I saw some interesting types and I decided to write about them. Understanding the nature of your defects could prove to be useful for your day-to-day work.Most of the program defects that I have seen fall into one of the following categories:

  1. Asking a component to do too much. Closely related: having only a few really big components in a large program.
  2. Misunderstanding the responsibilities of a component, or never deciding on clear responsibilities in the first place.
  3. Not knowing the cost or nature of operations, language features, library, API calls and thus negatively impacting program performance.
  4. Misunderstanding what is global data and what is local data and as such synchronizing too much or not enough.

Let’s examine each of them in more detail.

1. Asking a Component to do Too Much

It probably happened to you before: you look at the list of methods from a component and you can’t tell what is this component responsible for. As you read through the list, you get the impression that this component is universally usable for about a half or a third of what the program does. Unless this component is a facade for a subsystem, you are in trouble.

You also notice that it is increasingly difficult to debug the system. It feels like all roads lead to Rome, or translated to software: no matter what call you are tracing, you have to pass through the same giant components. After a while you don’t understand where the calls go and you find yourself just trying things out.

To make sure that we are on the same page, here are the definitions for a component and for a class: A component is a concrete specification of an abstraction (either from the problem domain or from a supporting domain, commonly known as API or library). A class is a concrete specification of an abstract data type (ADT). How do components and classes relate? A component is made of one or more classes. The key here is that a component usually has a real world equivalent, while the class is usually a “fabrication.”

This is a good time to stop, reorganize, maybe redesign, and refactor. Figure out how can you make your component more cohesive again. Pare down the responsibilities to a set that makes sense and is cohesive. Maybe create a few new components. As you are reworking the components, watch out for coupling between them. It makes sense to draw up a dependency graph and to get rid of loops from it.

Maybe the way you notice this defect is that you are not sure where to put the new functionality that you have to add to the program, and you start putting things in components pretty much randomly. This leads into the next problem:

2. Misunderstanding the Responsibilities of a Component

There is a quick test to see if the responsibilities of a component are clear: read them and then see if you can retell them. If you remember what a component does, then it is easier to add a new responsibility that is a good fit with the existing responsibilities. If you don’t remember what are the responsibilities of a given component, then you are almost guaranteed several program defects both by you and by others who use the component you have created. The same applies to method names, requirements, etc. This test has not failed me yet.

If you organize the program, as much of it as you can, into components that represent aspects of the customer’s business, then your program structure will be more resilient to changes then if you organize it along features. The customer’s business fundamentals tend to not change nearly as fast as their wishes about your program. And when the business changes, your changes might be localized to the business components.

3. Not Knowing the Cost or Nature of Operations, Language Features, Library, API Calls and thus Negatively Impacting Performance

In the The Design and Evolution of C++ Bjarne Stroustrup describes one of the Low-Level Programming Support Rules for C++ as: what you don’t use, you don’t pay for (zero-overhead rule) (121). The interesting part of this statement is not explicit: you must know the cost of each language feature or you won’t be able to decide what you are willing to pay for.

It is interesting to note that Stroustrup designed C++ to suit his needs and the needs of its users. Chapter 4 contains the C++ Language Design Rules. He had a problem with “how to analyze the network traffic [in telephone switching equipment]” and he decided that in order to solve this rather complicated problem he needed better tools. So he created the Cpre, which later became C++.

Amongst you the Java programmers may wonder how this applies to you. Quite well. Even when you program in Java you need to know what is happening behind the scenes if you are to write production quality software. Otherwise, your program will be plagued with performance problems.

Performance problems are an ugly beast because they don’t show up when you test your program with a few users or limited data. Everything seems to work fine and takes a limited amount of time. But then you turn on the firehose and all things come to a halt. The display routine doesn’t work anymore because handling 2,000 items is not the same as handling 10. The database is slow, because the query that worked on 100-record table doesn’t work well on 10,000. I know that some of you are fanatics about this stuff and have solved this problem well. Drop me a line and share your experience with others.

OK, you say: What is the answer? Well, it depends. I think that knowing that this problem exists is half the battle. The other half is devising the right solution. Tell me about your specific problem and I will help you come up with a solution.

4. Misunderstanding What is Global Data and What is Local Data and Synchronizing Too Much or Not Enough

One of the basic rules of concurrent programming that I learned back in college that still holds true in the age of Java is that you must have protected access to all global data. This is a simple and easy to remember rule that can keep you out of a lot of trouble. Now all you have to do is keep thread local data and global data separated (don’t forget about those parameters that get passed back and forth between threads) and you will have yourself a well-behaved program.

This is not nearly as hard as it sounds. If you “limit” yourself to only doing things this way, you might not be writing some potentially exciting component that solves a problem in a completely unique way that nobody else can get without two PhDs. But, hey, the program you write will work. Period.

Here are a few simple steps to help you figure out what global data is and how to deal with it:

  1. Count the number of separate threads in the program.
  2. Count the number of threads that can run at the same time.
  3. Identify all pieces of data that are accessed by more than one thread identified in 2. Include parameters that get passed between threads in this list. You must protect access to this data.
  4. Protect access to data with locks. Use one or more locks, as necessary. Don’t forget: Always attempt to lock in the same order and unlock in the reverse of the locking order.

Again, this is a minimal solution, but it works. You can solve most problems this way. If you think that there is anything wrong with it, drop me a line. For those of you programming in Java, a good book on the subject is Concurrent Programming in Java, 2nd. Ed., Design Principles and Patterns, by Doug Lea.

I know that both of these subjects deserve way more detail than what I have time for in this short note, but I hope that now that you are aware of these problems, you will take steps to prevent them in your next project.

How Can You Start?

A few simple rules can get you going and can help you create better components:

  • Component interfaces should be minimal yet usable.
  • Class interfaces should be primitive.
  • Public interfaces should be complete.
  • Private interfaces should be sufficient.

Ask the question: What component would I need to have to get the job done? If I would have a component that can do x, then could I implement what the customer is asking for? and so on. Group the requested features into components. Do the memory test with them. Ask a friend to also do the memory test. In no time you’ll be writing great components.

For more on this refer to the Lakos book, Large Scale C++ Program Design. And don’t worry about the title, most of the stuff applies nicely to Java as well. Another good source for advice and thorough theoretical explanation is Object-Oriented Software Construction, 2nd Ed. by Bertrand Meyer. And of course, you can always crack open Code Complete by Steve McConnell.

Let me how your components are coming along.