Practical Software Engineering

In the software business where most people don’t seem to agree on anything it seems that everybody agrees that planning is hard. (The only thing that seems to be even harder is tracking to the plan and regularly replanning, but that’s another story.)

Why is it that we have such a hard time with planning? Our thinking about the goals and benefits of planning seem to stay in our way. Here are four mistaken beliefs that many folks have and what can be done about them:

1. “It is not worth planning, life’s changing too fast.” Life is changing fast, but that doesn’t mean that it is not worth planning. It just means that we need to evolve the plans as things change. A plan at best represents your thinking in a moment in time. As you learn more, you need to improve on your plan.

2. “The only reason to plan is to figure out what the end date is, but they always give me the end-date anyway.” “They” might be the management, client, architect, team lead or that “other” team; you pick. Knowing the end date is good, but you still don’t know what’s included. Of course, you respond to this that: “they” also tell me what I have to build. However, “they” are unable to specify how much of those things to build. That is still up to you. It is up to you to figure it out how to make it happen. If you cannot find a way, at least you can tell “them” early that you will not be able to make this happen. And, you will be be able to back up your statement with a detailed plan that shows that you had thought through the problem.

3. “I cannot create a perfect plan.” Nor can anybody else. You need a plan at the level of granularity that can help you perform your work. You are creating this plan for yourself (or your team), hopefully not for others. Create something quick, then updated it “early and often.” The plan needs to represent your best knowledge at the moment you created it. Accept the fact that you will learn as the project progresses. Learn to live with incomplete data. These are key to your success as a software engineer.

4. “I don’t know how to plan. I am not given the opportunity to go to a class to learn it.” Nobody was born with perfect planning skills. You may have the natural talent to think through problems with incomplete data (which helps), but everybody can learn the mechanics of planning and everybody can become pretty good at it. It is your responsibility to seek out learning opportunities. Let’s face it: nobody has your interests at heart more than you do. So, it is your job to find ways to improve your skills. Once you learned the basics, then keep practicing.

To sum it up: A plan is a roadmap to project completion. Just like any other map: “when the map and the terrain don’t agree, trust the terrain” and update the map. Planning is an aquired skill. With practice you can get good at it and you will derive benefits to make it worth learning it.

To convert the unknowns into knowns, you need to aggressively prototype.

What to Prototype?

Here are some examples:

Prototype the use of an API. Create a prototype to see if the API that you need to use works the way you think it does. This includes confirming that it can be called with parameters of the type and range that you can provide. The API may belong to a component that one of your team mates wrote, or it may belong to a third party library. Either way, you want to be absolutely certain about how to use it. No amount of documentation reading will substitute for a well pointed prototype.

Prototype an algorithm. Create a prototype to prove to yourself beyond reasonable doubt that you can implement the solution to a problem.

Prototype an interaction. Create a prototype to show that the interaction sequence or method that you want to use in your application actually works for the intended use. You can take the prototype and usability test it, incorporate the feedback, and then place it into your application.

What should the prototype look like?

Since we are talking about software development, your prototype should be a software program. A small program. As small as possible. A program that answers one or two critical questions and that’s it.

The prototype should be like the small program that you write to prove to the API writers that their API has a defect. It has to be small and prove one point. Your objective from the prototype to learn something that you didn’t know, or confirm something that you suspected.

How to Prototype?

Pick one or two key questions that you want your prototype to answer and focus your small program answer them. Keep in mind common software engineering and experimentation principles as you decide on how to proceed:

1. divide & conquer: write multiple prototypes as needed
2. reduce scope: keep the scope of each prototype to a minimum
3. isolate concerns: separate the concerns so that you can establish root cause
4. confirm behavior: confirm that an API behaves exactly the way you had thought that it would under the conditions that you anticipate
5. validate assumptions: validate that what you think is true, it is indeed so
6. validate invariants, pre- and post-conditions: half the battle is to identify these; you get extra credit for validating them

I have noticed that after a while some prototypes that started out with the best of intentions, become monsters. Don’t be tempted to stick another one or two topics into an existing prototype. Your prototype will become so complicated that you cannot confirm root causes for outcomes. Then you lost your ability to know things for sure.

Lastly: the most important rule is a well worn one: KISS=Keep It Simple & Sufficient.

Every once in a while I am surprised by somebody who says that prototyping is such a waste of time. For quite some time I have taken for granted that prototyping is a fact of my software engineering life. Maybe it is worth asking: what is the problem for which prototyping is the answer? Or, in other words: why should you prototype?

You should prototype when you don’t know enough about about the problem. Which is almost always. When your knowledge is lacking in some aspect, then it is time to write a prototype. When you don’t know enough about an API, some requirement, the operating system that you are designing for, the libraries that you want to use, and so on. When you feel that you have to answer a few clarifying questions, then it is time to answer them with a prototype.

The short answer is that you prototype because you don’t know enough to proceed with the project at an acceptable level of risk.

Almost ancient history now: it was in 1993 when Microsoft came out with OLE (Object Linking and Embedding). OLE was based on a simple premise: a component can ask another component a simple question: “Do you support interface X?”

Let’s pretend that I am a component and I found you (another component). I would be asking you to see if you support any of the interfaces that I need. If you do, we can work together. If you don’t, I have to keep looking for a component that supports what I need. One thing complicating the situation is that you have to support the version of the interface that I can work with.

Interface versions bring us back full circle to talk about supported versions and version ranges. Without versioning and the proper assignment of version numbers it is difficult to make component-based software development work.

Life, especially software development life, is never without constraints.

Time Constraints

1. The software has to be released on a certain date, set well in advance. The only option is to plan based on the time budget.
2. Given that I have this much time, what can I fit into this time?

Software Correctness (or Incorrectness) Constraints

1. The software that you work with has flaws in it.
2. How many flaws?
3. Are the flaws of the software making it unusable? This is not any different than any other software you are trying to use. Windows has flaws in it, however once a problem is discovered, you are always trying to use the program in a way as to avoid that particular flaw.

Incomplete Knowledge Constraints (Requirements -> Problem -> Solution)

1. The understanding of the requirements will evolve as you move through the project. More so, even the understanding of the folks who specify requirements will evolve.
2. Your understanding of the problem evolves.
3. Your understanding of the solution evolves.

Each constraint requires a different treatment. Each poses risk, therefore each needs to be dealt with, or the project will get into jeopardy. The questions to ask about each are:

Time Constraints

1. What can I do in 2/20/200 hours?
2. What % of my time should I spent on a given activity?
3. What is the activity that only I can do?
4. What is the activity that I can do that is visible?
5. What are those activities that I can do to reduce other’s dependencies on me?
6. What can I finish now to unblock to others?

Software Correctness

1. Do I have proof in form of running code that this API I want to use works in the way I want to use it?
2. Given that I found a problem with this API, is there a workaround that I can use?

Knowledge Constraints

1. What are the requirements that are at risk to change because of real or perceived lack of clarity/definition/etc.?
2. Is the problem formally described? If not, why not?
3. Is the solution presented on paper proven to be correct (empirically or mathematically/logically)?
4. Does the solution provide flexibility to cover for the risk of changing requirements?

People’s Skill and Knowledge Constraints

1. What are the skills available in the team?
2. What are the growth goals for the people on the team?

There is no constraint-free state. Our job as software engineers is to figure out a solution that can be completed within the constraints of time, people, money, etc. that we have. It is not an easy task, but that’s what makes it fun.

Well, Peter Drucker didn’t write about software management, but he did write about management. In his book Managing in a Time of Great Change he has a chapter called The Five Deadly Business Sins. Here is what Drucker has to say:

The third deadly sin is cost-driven pricing. The only thing that works is price-driven costing. Most American and practically all European companies arrive at their prices by adding up costs and then putting a profit margin on the top.

The only sound way to price is to start out with what the market is willing to pay—and thus, it must be assumed, what the competition will charge—and design to that price specification.

How does this apply to software development? Here is how I see the two scenarios in the world of software development:

Cost-Driven Pricing: A Tough Sell

In this scenario the Customer comes with the software product requirements (not quite complete, since they never are). Next the software team goes ahead and figures out how long it is going to take to build it, and tells the Customer what the expected end-date is, thus arriving at a “cost.” The Customer is almost always unhappy with the end-date that the team provides. The Customer is also unhappy with the delivered product, since the software team usually does not manage the scope at a fine-grained level. The team thinks that they know what they need to build, and proceed on building it.

Price-Driven Costing: Problem Solving with Constraints

In this scenario the Customer comes with the requirements (never complete), and a target end date in mind: an end-date that fits into the business scenario that they intend to play out. It is now up to the software team to:

1. Figure out how the product can be built under the given constraints of date and cost, and
2. Figure out how to “slice & dice” the requirements to meet the Customer’s goals.

The software team must deliver to and negotiate with the Customer regularly. To deiver the project the team have to bring their ingenuity to solve the problem.

The Bottom Line

Price-Driven Costing applies to the way software teams work. It means that we need to work back from the price (date) that the Customer is willing to take, and then we have to figure out how to deliver the project on that date. There may be times when we have to say, this cannot be done. And there will be others, many others, when we will successfully deliver the project to a happy Customer.

Summary

1. The purpose of the design and code reviews is to find defects.
2. The factor that most influences the length of a design or code review is the amount of work product that has to be reviewed. This may be pages of design or lines of code of source.
3. Checklist-directed reviews are the most effective at finding defects specific to the project and product. The checklist has to be built from past defects that the person or team had problems with most recently.

Starting a Review Practice in a Team

Checklist-directed reviews have a 70% to 90% yield in finding defects. The yield is defined as the percentage of defects found by the review that existed in the work product on entry into the review. When a team starts a review practice, there is no checklist yet. How can a Team get started? Here are some tips to get the reviews going.

What parts of the system to target for a review?

1. Create on a large piece of paper, or on a whiteboard, a map of the system that the team is working on.
   1. This map must include at least all the physical modules of the system, but going to a finer level of detail is better.
   2. Each system part drawn should represent around 2 KLOC of NCSS (KLOC=thousand lines of code, NCSS=Non Comment Source Statement).
2. Review the defect log.
   1. From the defect log select the most recent list of 100 defects (or whatever you have in the last month).
3. Place a mark by each system part for each defect that can be traced to that part.

The system part with the most marks gets to be the target of the investigation.

If after a cursory assessment it seems that finding and fixing all the defects in the part would take longer then rewriting it, then the team should choose the rewrite route.

How much code can you review?

The speed of the review seems to be constant on all projects. Ranges from about 200 to 300 NCSS per hour. Reviewing at higher speeds will make the reviewers miss what they are looking for.

What is the difference between a review and defect finding?

During the review the reviewer is looking at the source code, and when he/she notices a defect he is looking at it right then and there. There is no more detection needed. On the other hand, when the developer has to investigate a defect that is reported by a Customer, then first the reason for the defect must be established. This is a time consuming activity because it contains a great deal of uncertainty.

What to look for in a review?

It is best to look for defects that had been observed already. Chances are that the defects that were noted in some part of the system, are present in some other part of the system as well (especially if they both have just been worked on).

Types of Reviews

1. Personal Review
2. Peer Review
3. Inspection
4. Walkthrough

Review Principles

Personal reviews follow a process with

• entry and exit criteria
• a defined review structure
• guidelines, checklists, and standards

The personal review goal is to find every defect before the first unit test. To address this goal:

• use coding standards
• use design completeness criteria
• measure and improve your review process

Design Review Principles

1. Produce designs that can be reviewed.
2. Follow an explicit review strategy.
3. Review the design in stages.
4. Verify that the logic correctly implements the requirements.

What is a Reviewable Design?

A reviewable design has:

1. defined context
2. precise representation
3. consistent and clear structure

This suggests that:

1. the design’s purpose and function is explicitly stated
2. you have criteria for design completeness
3. the design is explicitly structured in logical elements

Checklists

Checklists: The Theory

1. When performing precise tasks, it is difficult to do more than one thing well at a time.
2. The checklist defines the review steps in the suggested order for performing them.
3. By checking off each item, you are more likely to perform it properly.
4. Establish a personal checklist that is customized to your defect experience.
5. Process yield is the percentage of defects found before the first unit test execution. (70%+)

Checklists: HOWTO

1. Use your review strategy.
2. Review one product component at a time.
3. Check for one type of defect at a time.

Checklists: The Key Point!

Treat each check as a personal certification that the product is free of this defect.

For Extra Credit: Estimating Remaining Defects After a Peer Review

Use the Capture-Recapture Method:

• A: The number of defects found by the first reviewer.
• B: The number of defects found by the second reviewer.
• C: The number of defect found by both the first and the second reviewer.

Estimated total number of defects in the product: E = A * B / C
Total defects found so far: D = A + B – C
Estimated remaining defects in the product: R = E – D = (A * B / C) – (A + B – C)
Inspection Yield: Y = D / E * 100

HOWTO Quick Summary

1. Pick a piece of code that you are uneasy about.
2. Review it for defects that you had recently. KEEP A LIST!
3. Keep your review speed at 200 LOC/hr. Plan ahead!
4. Only look for one defect at a time.
5. Treat each check as a personal certification that the product is free of this defect.

See Also

Guiding Principles for Reviews from Wiegers

1. Check your egos at the door (Weinberg)
2. Keep the review team small.
3. Find problems during reviews, but don’t try to solve them.
4. Limit review meetings to about two hours.
5. Review only about 200 to 400 NCSS per hour.
6. Start the reviews where the perceived pain is the greatest.
7. “It is only a mistake if it gets out of the review.”
8. “Avoid using technical reviewers who are themselves ‘above’ review.”

References

1. Peer Reviews in Software, by Karl Wiegers
2. Introduction to the Team Software Process, by Watts Humphrey
3. When two eyes aren’t enough, by Karl Wiegers
4. Seven Thruths About Peer Reviews, by Karl Wiegers
5. Seven Deadly Sins of Software Reviews, by Karl Wiegers
6. Handbook of Walkthroughs, Inspections and Technical Reviews, by Freedman and Weinberg

Business and society operates on commitments. For your commitments to be more than just empty statements, you must base those commitments on a plan of action. Creating plans is activity that can be learned. It takes practice to become good at planning.

Sage advice: because the business can only operate on commitments, commitments will be made. If you don’t create a plan of action, then the commitments will be made without your input, but you will still be held responsible for making them. It is in your best interest to be actively involved in planning and tracking your work so you can make real commitments that you can keep.
Guidelines

Start with the end in mind!

Start with the end in mind is a statement made popular by Stephen R. Covey’s book, “The 7 Habits of Highly Effective People.” This applies to software development very well. Think through your objectives for the project, and figure out what are the deliverables that you have to produce to be done.

Identify your work products: The Deliverables

Think through and answers the following questions:

1. What is the end result of your work?
2. What is your “value add” on the project?
3. What is your contribution to the project’s success?
4. In what order do you have to create your work products?

The answers to the above questions will clarify to you what you have to do on the project.

Identify the milestones

It is a mistaken assumption that because you don’t produce “running code” as a result of what you do, therefore your deliverable is not that important, and therefore it doesn’t have to be planned and delivered according to a committed schedule. This thinking is dangerous. Your deliverable, the work product that you produce is an input to some other person’s process, and if you are unable to create your deliverable, then the person counting on your output will not be able to produce their deliverable. Your work product might be a: presentation, white paper, design diagram, UI sketch, etc. Whatever it is, as long it is an input to somebody else’s process (or to your own process a few days down the road), you need to ensure that it is delivered on the agreed upon schedule and with the agreed upon quality.

Unless you are working on a simple and quick deliverable, say one deliverable that takes 8 to 10 hours to create, do not try to make the deliverable in one shot: “Divide and conquer.” Take your work product, and figure out what are the intermediate stages in which it has to go through to be created. Identify what could be its constituent parts, and in what order should you create those constituent parts. Then figure out the tasks you have to do to make those parts and then to assemble your work product.

Identify the work

A task list is nothing more than the work that you know that you have to do to get a set of deliverables done.

What will you do first? What will you do next?

Let’s suppose that it is Monday morning, the first day of the new project. You arrive to the office early to get ready for the new project. What will you do first? Write these few things that come to your mind down on your task list. Write down your first few tasks as you would do them on the first day of the project. You would have to think about what will you do anyway, wouldn’t you? So, write them down. Then think about the next milestone. How are you going to achieve that? What will you have to do? You will do … … And then what? Write that down, too. Your goal should be to write down everything that you have to do to make the project work.

Write tasks based on what you will know when you get to start the task not what you know now

At some point as you write down your tasks you encounter the familiar feeling: “Yeah, but… I do not know what will I do after getting to xyz.” What are you supposed to know at the time when you get to that point? You sure will have learned some new stuff. If you wouldn’t be planning on learning anything, then you would have to use your existing knowledge and that means that you should be able to use all the knowledge that you have now.

What do you expect to learn as you work through the tasks of the project? Based on what you expect to learn, now, as you are writing the tasks, you can assume that when you will get to that point in your project, you will have learned what you planned to learn, or you have replanned the project, because you realized that you are not able to learn what you need to.

Of course, the assumptions that you make should go on your Personal Assumptions List. This may also be a new thing for you, so keep the assumptions list in writing. Remember: the problem with assumptions is not that we make them–after all we make them all the time–but rather the problem is that we make them and we never realize when an assumption “changes sign,” or goes from true to false. Every time when you notice that an assumption is going south, then you need to update your plan, since part of your plan is no longer true.

When you think of a new task, write it down!

This may sound obvious, but it is not. It is much better to have captured more tasks and then throw some out, than far fewer then you need. The insight about how to do something might come to you when you least expect it. Write it down! Don’t count on the “great” insight to come again and again.
Update your task list early and often!

The task list must represent at all times your best understanding of what needs to be done to complete the project. While it is important that you keep a history of the task as you started out with, it is even more important to always know what comes next and what is left to do to get the work done.

If you catch yourself that you are working on something that is not on your task list, then add the task to your list. The task list serves two purposes: it show what you expect to do, and–through the time log–it shows what you actually did.

Log time against a task that you planned to do

Logging time is considered by many folks–especially software folks–one of the most difficult things to do. Logging time is easier if–whenever you ask yourself the question “What do I have to do?”–you turn to your task list and start working on your next open task.

Twice a day, at mid-day, and at the end of the day, check your time log and ask the following questions from yourself:

1. Did I do what I planned to do?
2. How much time did I log?
3. Is there a good reason for having logged only this much time? What is the reason?

Understand the reason for the variation

By itself the data that you get about your planning and tracking process is very good. But it is even better to look at the variation of your data from day-to-day and from week-to-week. Seek to understand the reason for the variation. If there is no variation, but you are not happy with the results, then look into how you could make some changes that would cause a positive improvement in your data.

To sum it all up: At first planning is hard. With practice you can get pretty good at it. Your corporate life will be better when you will become known as a person who can make and meet commitments.

Metrics! Schmetrics! Are we afraid of metrics?

The Austin Software Process Improvement Network meeting I attended tonight was a roundtable on metrics. As I was listening to the panelists explain their experience and ideas about measurement, it occurred to me that most people harbor some basic fears people have about measurement. In this article you’ll read about the three basic fears that I think people have and you’ll find some ideas for dealing with those fears.

In order to measure anything at all, I suggest that we have to overcome these three nagging fears:

1. Fear of Self: I don’t want to measure anything because I don’t want to know how bad/good the situation is, since I intuitively know that it is bad/good.
2. Fear of Others: I don’t want to measure because I don’t want you to know how bad/good the situation is, and it is none of your business anyway.
3. Fear of Change: I just like what I am doing–even if it is not very effective–and I don’t want to measure anything, because if I do, I might have to change what I am doing as a result.

1. Fear of Self

This fear could be coined the AA Syndrome (Alcoholics Anonymous). As long as I don’t own up to the fact that some problems exist, I don’t have to deal with it. I heard this problem expressed by somebody this way: “You cannot get one to solve a problem that one doesn’t have.”

This fear is possibly the hardest to deal with. There is not much you can do about this fear in the short term. It is one of those fears that if it is present in the person, then at best you can get them to change their mind about it in the long term.

The people that have the internal drive to continually improve generally know how to overcome this fear. They look at the measurements that they get as opportunities to improve themselves and their personal and professional lives. The people that have the desire to better themselves tend to be self-disciplined. They posses an inner drive that far surpasses any leverage that you may have over their behavior. I feel like this is a quality that you have to look for when you hire.

2. Fear of Others

Most of us are afraid of being measured. Why? One of the reasons is that we have been measured, tested, and evaluated by others so many times–often in less than useful ways–that over the years we came to trust the people doing the measurement less and less.

3. Fear of Change

To compound the problem, we are afraid of metrics even if we do the measurement. We have a hard time owning up to the fact that we ourselves are the primary reason why ………… (substitute your favorite problem here). This problem may lie at the heart of many of our other problems. As Peter Senge puts it in The Fifth Discipline, we have a difficult time learning when the cause and effect are separated in time and/or space. Getting some metrics is a key element of making that all-too-important connection relating the cause to the effect. While teaching the Personal Software Process class I noticed this pattern over and over again.

It is difficult to grasp the relative value of the things we do and their contribution to the overall results that we are trying to achieve. Let’s take the first example from software development: when software developers take the Personal Software Process class they have a certain belief about what works and what doesn’t, and what is the cost of a certain development activity. As soon as they start to measure what they do, the world changes… They realize that some of the practices they thought are wasteful turn out to be very productive, while other practices don’t contribute nearly enough to the overall results.

How can we overcome our metrics fears?

Can any metrics initiative overcome any or all of these fears? I don’t know. I would say that if it doesn’t overcome some of these fears, then it will be doomed. While I cannot give you a recipe for success, at least you know three problems that will popup and you can prepare to deal with them. It’s a start.

What is your metrics story? I’d like to hear it.

Definitions

• “A metric is a quantifiable measurement of a product or process.” –Linda Shafer
• “Quantities are measurements of qualities.” –Paul Kirchner

You probably have to. Knowledge workers today make up over 70% of the working population. Your productivity is your competitive advantage! Take charge of you future! Discover what makes you more productive, and put yourself on a trajectory for greater success.

As a Software Engineering Manager, the performance of software developers is on my radar every day. You owe it to yourself to understand and improve your software development performance. Think about your performance. Today. And do something about improving it.