Tag Archives: Design

Master Your Software Development Tools

Like all professionals, software developers built themselves tools, many software tools, to help them get their job done better. They used other tools, too. The chalkboard (replaced by the whiteboard), and paper & pencil have been with us from the very beginning. People wrote the algorithm, converted it by hand into machine code (in octal at first, then hexadecimal), and input it bit-by-bit into the machine. Later, as compilers were developed, the compiler became an essential tool. There are more tools now in the developer’s toolbox, each serving a unique purpose in the software development life cycle. With each tool comes the need to learn how to use it, to become good at getting the work done with it.

As a software engineer you are designing and implementing software. You, like most people, might start out with a sketch on the whiteboard or on a pad of paper. You sketch out what the system will look like, sometimes the UI, sometimes the internal structure, sometimes how it will interact with other systems. These sketches can be invaluable in conveying to the rest of your team your ideas. So practice sketching. You don’t have to become an artist at drawing. Just get good enough so that others can understand what your drawings mean and can build upon your ideas. And make sure that your writing is legible. Your good ideas will be lost if a day later you can’t read your own handwriting.

Most likely not all of your team will be in the same location, so you must take at least a picture of your sketches and share them with others. Or, you may need to draw them using a drawing, sketching, or diagramming tool that aids you in the capturing of designs. The more popular diagramming tools are Visio and OmniGraffle, but there are many others, too. You can also use a presentation software as a diagramming tool, since many have decent drawing capabilities. Either PowerPoint or Keynote will do. An open source alternative is Apache OpenOffice, which has a drawing tool and presentation software as well. An added benefit is that it also runs on many Linux variants. Some diagramming tools maybe already built into your favorite online tools, like draw.io, Gliffy, or Lucid Charts. Another popular online option is Google Draw. Many new tools are popping up every day.

Of course, you’ll also need a code editor. Pick an editor or an IDE and learn it well. People who have been in the software world for a long time tend to gravitate toward a few favorite editors: emacs, vi, TextMate, Sublime Text, Atom, or towards some of the popular IDEs: IntelliJ IDEA, Visual Developer Studio, Eclipse, Oracle JDeveloper, Xcode, NetBeans. Sometimes your workplace or your project determines which editor or IDE you have to use. Then you need to learn that tool, too. It is worth spending a few extra hours learning the tool that you need to use day in and day out—instead of fighting the tool every day. This is true for all other tools you have to use as well, not only code editors.

Set a personal goal to learn how to accomplish the most common tasks with keyboard shortcuts with just about every tool you frequently use. Write on a Post-It note 5 shortcut keys, and keep it in front of you. When you mastered those, add 5 more. As you get more proficient in the use of your tools you will notice that you can focus on the task at hand, instead of focusing on the tool or how to get a particular task completed. Once the tool blends into the background your performance will improve and others will start to notice the improvement as well.

Every now and then one still bumps into a developer who doesn’t deploy their code to see it running after he or she made changes to it. The basic process that all developers need to be able to perform on every project is at the minimum: compile, build, install and deploy, and test. There is no substitute for seeing your own code run and do what it is supposed to do, regardless of how confident you are that the one-line change or new feature you just implemented works. At the beginning of every project take the extra hour or even extra day that you need to ensure that you figured out how to run the code that you are about to work on.

Write a “Hello, World!” application for the platform that you work on, and make sure that you have automated your development loop. The work of compiling, building, installing, and testing of the code should be automated. Automation ensures that the build steps are performed consistently, so when there is an error it is easier to find the cause of that error. It allows you to focus on writing code for the desired new features and not focusing on the mechanics of the work.

Just as much as you are writing code, you will also need to fix it, too. You need to learn to use a debugger for the platform that you are working on. Even if at first you don’t have to do this, it is a good idea to learn how to debug your code, so you don’t fumble with the debugger the first time you really badly need it, because you will need it. Besides, you probably should consider stepping through code in the debugger from time-to-time, just to ensure that you know what’s going on when the code is executed.

Using revision control can be a mystery to new folks entering the field of software development. Why would you want to use it? Couldn’t you just have regular backups? Revision control is not a substitute for backups. Revision control is the mechanism to help you write better code by isolating the additions, deletions, and modifications you make to the codebase for each meaningful unit of change, be it a feature or a defect fix. Revision control helps you work as a team, where there are many others who also work in the same codebase. These days (late 2016) Git seems to be the revision control tool of choice for a lot of companies and projects. You should learn how to use Git, and get really good at it.

In addition to a tool, you also need a revision control process. Gitflow is one such process. As you work, adapt the process to your and your team’s needs.

Software development projects extend over several weeks or months and as such, guiding them toward a successful end is a significant effort. To assist in this effort, every project needs a shared task list to plan, guide, manage, track, and report on the work. A such tool is a centralized issue management system. “Issue” is a generic term; it can be a feature, a trouble ticket, a task, or any other work item that someone on the team needs to work on. The issue tracking system can assign different workflows to each type of work item to ensure that each follows the appropriate process toward its resolution.

Jira and Bugzilla are popular issue management systems. They even have connectors into revision control tools that allows the user that opens a work item to optionally create a code branch for the new code associated with that work item. GitHub Issues is the issue tracking system integrated into the popular GitHub revision control tool.

As you work with an issue management system, keep in mind the following:

  • Once you write up an issue, that issue becomes inventory and you have to manage it, which means: define, prioritize, validate, refine, develop, verify, and support it.
  • Keep the issues relatively small, meaning: each issues should require about 1 to 2 days of work to complete. Following this rule, you will have about 5 to 7 issues for a week. This allows you to maintain a good pace for your work, and always able to report where you are.

Here are a few more tools that you should know about. Each can help you get your work done better, faster, more effectively:

  • Code review tool that provides a collaborative environment for code reviews. Some revision control systems have such capabilities, but a specialized tool will improve your team’s output.
  • Unit test framework to run all unit tests. Some of these capabilities are part of almost every modern build system.
  • Unit test code coverage tools that measure how much of the product source code base is exercised by the set of unit tests your team wrote.
  • Code analysis tools to identify dependencies, measure code size, cyclomatic complexity, etc.
  • Requirements capture and management tool. The wiki or the issue management system can do this job, too.
  • Code performance measurement tools. Some of these are capable of estimating power consumption (which is very important on mobile platforms).

You owe it to yourself to learn the tools that you use every day to get your work done. By getting proficient with the tools you use, you allow yourself to focus on the task at hand, and sooner or later your tools will blend into the background, allowing your creativity to shine.

Enjoy the journey!

Software Engineering Essentials for the Working Software Developer

The previous article in the series mentioned topics that are rarely if ever discussed in a Computer Science curriculum. The topics in this article are also often skipped. Unfortunately, the knowledge of these software engineering topics is quite important for a working software developer who has software to build and deadlines to meet.

This the fourth article in the series on Starting in a Software Development Career.

Structure: Components, Modules, and Subsystems

The previous part of this series, on Computer Science Essentials, started out with the importance of assigning responsibilities to parts of the software system. But what are those parts? Here is a quick rundown of some terminology to think and reason about the structure of a software system.

This terminology was born out of the need to enable a team to build a reasonable size system, between 100,000 to 500,000 lines of code. Anything beyond that size should be considered “a system of systems.”

In most programming languages the simplest structural software element is a statement. However, program language statements don’t stand alone, they are grouped into a function. Functions are grouped into a class, struct, or namespace depending on the language that you are using. Classes are grouped into a component; components form a module; modules form a subsystem, and—at the highest level—subsystems form a system.

The structural elements above are only a recommendation, a rule of thumb. Once adopted, they imply a certain size and level of abstraction for the team that is using them. You may have to customize this for your situation. Start by establishing a hierarchy of parts and target sizes for each structural element. Here is a recommendation on sizes for you to start with:

  • Function: 10 lines of code (LOC); range: 2 to 20 LOC
  • Class, Struct, or Namespace: 200 LOC; range: 100 LOC to 1 KLOC
  • Component: 2,000 LOC (2 KLOC); range 1 to 4 KLOC
  • Module: 20 KLOC; range 10 to 40 KLOC
  • Subsystem: 50 KLOC; range 20 to 100 KLOC
  • System: 250 KLOC; range 100 to 500 KLOC

The objective of establishing the nomenclature for the levels of abstraction and sizes is to speed up the conversation about the system. This provides the vocabulary for a team to talk about the system.

Why should you care about structure? What is the meaning of these structural elements? The function and the class are basic abstractions, and they extend the capabilities of the language and its fundamental libraries. The class is an abstract data type that extends the language. Components, modules, and subsystems group related functionality in the problem domain. These elements enable the team to quickly talk about these without getting into the details of their implementation as they are composing parts and building new functionality.

Revision Control

When building software some of the paths that you and your team go down on will be in error. You will have to backtrack and restart work from a known good state. This is where a revision control system comes to your rescue. In a nutshell: the revision control system allows you—the individual developer—to carry out experiments, recover from mistakes, and—for your team—to share the work with your collaborators: allows you to co-create.

In order to benefit from revision control, you have to make your thinking explicit, and use the revision control system to record your decisions. For example: you just received a defect report that states that the shipping charges are not computed correctly. Next you create a branch, figure out what the problem is, implement the fix, verify that it all works, then merge your branch into the main code line. If your fix fails, then you can restart your work from the point where you started from.

Similarly, create a branch for any new work you are doing on the system. Working in the branch creates a moment of calm and stability while you implement the new feature. The time you work in the branch should be short (a few hours to a couple days at most). Any longer than that and the moment becomes a frozen epoch, and you face a mountain of changes as you try to bring your new feature into a system that has evolved while you were not looking.

Issue Management

Issue management is one of the dark knights of software development. It is powerful, but often avoided by developers and their teams. Sometimes, it can feel like a lot of work. Issue management describes a group of related activities.

Define the work. Issue management is an expression that refers to the management of defects, tasks, features, or any other items of work that are collectively known as issues or work items. Think of an issue as a unit of work. Start by writing down what the work is; this becomes an “issue.” The issue needs to have at least two parts: a name, and a description. Find a good name that helps you quickly recall what this work is when you see the issue name in the list. Once you named the issue, next describe it in sufficient detail that you can recall what that work is even a week or two later. Now that you defined the issue, you can start managing it.

Order and schedule the work. You must decide in what order and when to do the work. The simplest method is sequencing issues one after another in the order in which they can be completed based on the dependencies between them. You could also schedule the issues on the calendar. Or you can use a more complicated scheduling algorithm that distributes the work across many people and based on constraints and rules. When you start out and your team is relatively small, just stick to the simple stuff.

Perform the work. Do them in the order that you arranged them and how you scheduled them. This sounds simple. What complicates it is that you need to be mindful that if you see that the defined work no longer makes sense, then you need to change it, and quite possibly you need to define new work. You may need to change the ordering and scheduling, too.

Capture the progress and the completion of the work. While you work, remember to record your progress. If something takes longer than an hour, set the status of the item to “In Progress,” especially if you are working on a team. After all, you don’t want two people to work on the same stuff. When you are done, mark it complete. It feels good to finish stuff!

Revision control and issue management should work together. Revision control systems provide a capability to associate the source code you write with an issue from the issue management system. This way you can tie the definition of the work to the work you did as a result of the issue you defined. This capability comes in most useful when you are trying to figure out how something went wrong.

Design

The hidden secret of design is that everybody says that they design, but you find almost no artifacts of the design activity, excepting a few confusing whiteboard sketches. Without artifacts, there are only thoughts about design.

Why no artifacts? One reason might be that is easier to talk about it, than to write it down. Design is a high-intensity mental activity. You have to envision how the system will look or work, before actually building it. You have to draw pictures and write some prose that talks about both the system’s structure and its behavior.

Here are the essentials: you needed to draw “boxes,” “lines,” and write some “text.” The boxes are software parts. The lines are dependencies between those parts, and the text explains their responsibilities.

The beauty and freedom of design is that starting from the top down you can invent the boxes that you need to implement your system. At the top level you divide up all the responsibilities of the system across the parts (the boxes) that you have identified. Draw the lines between them: this tells the story of how those boxes communicate between them.

Next, focus on one of the new “boxes” that you envisioned and the responsibilities that you assigned to it. How will it perform those? Design what’s inside that box. The method is the same: identify the boxes that are inside it, draw the dependency lines, and assign the responsibilities. No magic. Some part of the system has to do the work, that part must be identified, and the responsibilities assigned.

Whenever you can use “boxes” that already exist, so that you don’t have to write them. Then your job is to fill in the gaps, write those software parts that you don’t yet have.

Check your design by “role-playing” or walking through step-by-step the use case that the system has to support. Do you have a software part identified to perform every bit of the use case? Does the part have sufficient responsibilities to do the work? If yes, move on to implementation.

Architecture

Each of the topics discussed so far has several books written about it—this is true for architecture as well.

“The software architecture of a system is the set of structures needed to reason about the system, which comprise software elements, relations among them, and properties of both.” (#1) The reason you should care about architecture goes back to assigning responsibilities and making fundamental decisions about the system’s structure and desired behavior. These decisions will ultimately place constraints on the system’s implementation.

As you are starting out in your career in software development you’ll encounter architecture mostly in negative statements, like: “Someone needs to make architecture decisions around here.” and similar. Developers usually decry the lack of architecture, but rarely step up to the plate to actually do something about it. Architecture decisions must balance both technology and business strategy constraints. And this seems to be the stumbling block for most developers. A software system’s architecture must support the business strategy the organization decided to follow. However, most developers don’t want to get into business strategy, only technical strategy, therefore not that many people work on real architecture.

The need to focus on and the understanding of business strategy is also what distinguishes architecture from design. A system’s architecture in some sense also represents the totality of technical decisions based on the business strategy the organization adopted. Once the architecture is in place, then the design decisions that follow implement those decisions.

Just like with design, there is always an architecture for any system, even if no deliberate action was taken to devise one. This is fine sometimes, at least until the business makes some strategic changes. Then everybody wonders why things are the way they are with the system. As with some other activities, the value of architecture often becomes clear only when change happens.

Summing up: when you write software through a series of deliberate actions, you may still not get the software that your users want, but at least you will know how you got there. That gives you a chance to make the necessary corrections to get to what your users want.

Enjoy the journey!

  1. Bass, Len, Paul Clements, and Rick Kazman. Software Architecture in Practice. 3rd Ed. Boston: Addison-Wesley, 2013.

Computer Science Essentials for the Working Software Developer

Some key nuggets of knowledge toward writing good code are either not taught in school or are mentioned as esoteric theoretical knowledge and most people don’t realize how practical and important they are for competent, everyday software development work. Building software from components, assigning responsibilities to components, deliberately deciding how state is stored, coupling, cohesion, and cyclomatic complexity are essential elements of good software practice.

This the third article in the series on Starting in a Software Development Career.

When you write code you are solving problems: small ones, big ones, one problem after another. Problems you need to solve tend to be too big to solve with just a few lines of code. You have to take most problems apart, and identify smaller problems that make up the big problem (small ones that you can solve). These smaller problems are the components that you identify. To be successful at writing software that solves big problems, you have to be good at writing software components that solve a part of the big problem. Building software from components is the first critical skill to master for a software developer.

As you are taking things apart, identify which part of your code will do what. In other words, assign responsibilities to parts of your code (#1). A good component has clear responsibilities and collaborators to get its job done. Of course, as you are taking the problem apart, you must keep in mind that you will have to reassemble the parts in order to make the system work.

Each part of the software has to be responsible for doing some work, and you have to decide what that work is. Each line of code, each function has to be responsible for something, otherwise why would you include that line of code in the system? Assigning responsibilities to parts are possibly the most important decisions that you make during the design of the software system.

But why would you want to assign responsibilities to software parts? The short answer is to cope with complexity. This allows you manage the complexity of not having to think about how the part is implemented. As you are assembling the parts, what matters is the part’s interface and its responsibilities.

A subset of important decisions as part of the responsibility assignment are deciding which part of the system should hold on to state. State is a term that collectively refers to any data that the software holds. This state can be short-lived or long lived. Sprinkling the state across a large swath of the system will make it difficult to reason about the behavior of the system.

A good strategy is to make a large portion of the system stateless, sometimes referred to as purely functional. Make sure that this part of the system doesn’t have any side effects, in other words each function only operates on its input parameters and returns its results for others to use.

After graduating from college, like many others, you probably thought that most of what you were taught will not be all that useful during your “real” work. Interestingly enough—if you are a computer science graduate who develops software—most of the things you learned will come in handy at some point in your career, provided you know when and how to apply them. Here are three esoteric sounding things that are present every working day, even if you don’t recognize them: coupling, its close relative, cohesion, and their friend, cyclomatic complexity.

As you are deciding on the responsibilities and where to store state, you need to pay attention to coupling, cohesion, and cyclomatic complexity. Coupling is a connection or dependency between two software parts and it’s a measure of that relationship. Coupling exists, because it must, in every software system. Without coupling, or in other words, without being connected to or dependent on one-another, one function could not call another function.

A certain amount of coupling is necessary for a system to do its job. The problem appears when in a system there is too much coupling. For most cases it is best to have as little coupling as you can get away with. As systems evolve, the amount of coupling can become troubling. When there is too much of coupling it becomes difficult to change the system without creating undesirable changes to parts of the system that you didn’t intend to disturb.

While coupling looks at the relationship between parts of the software system, cohesion looks at what’s happening inside a software part. Specifically, cohesion describes how strongly the responsibilities of a software part (be it a function, class, component, module, subsystem, or even system) are related to one another, and how consistent those responsibilities are.

A software part is called cohesive if the software part’s responsibilities are focused around a single purpose and form a meaningful unit. Why is cohesion important? If we keep related things together, then we can reduce the churn caused by changes in the system. If you ensure that the software parts you write are cohesive, meaning that the related responsibilities are kept together, then as a result they don’t have to talk to too many components, thus they will have lower coupling.

So far we looked at a software part and said that it’s cohesive if its responsibilities are related to each other, and it has low coupling if it doesn’t have to talk to a lot of other parts to get its job done. Next let’s see how well it gets the work done. The formal definition of cyclomatic complexity is that it measures the number of linearly independent paths through a function or procedure.

Empirically you can get a feel for cyclomatic complexity by looking at the function. When you see complex conditional series of if .. else if .. else if, if .. and you get the feeling that this looks pretty hairy, then cyclomatic complexity is probably out of whack. There are tools that measure the cyclomatic complexity number (CCN) of each function in your code. If the CCN is greater than 10 for a function, then there is a good chance that it’s difficult to comprehend that function and you should consider rewriting it.

By paying close attention to the assignment of responsibilities to components, state, coupling, cohesion, and cyclomatic complexity as you are writing software, you can make your code much easier to design, write, and read. Keep in mind that you are the primary reader of your code, since you have to go over it many times before its complete. The easier you make it to comprehend, the more likely that you will complete it faster and with fewer defects.

Happy coding!

  1. Larman, Craig. Applying UML and Patterns, An Introduction to Object-Oriented Analysis and Design and Iterative Development. Prentice Hall PTR. Upper Saddle River, NJ. 2005.

Essential Software Design Documents

When it comes to documenting software designs, there are many practices ranging from a quick whiteboard sketch to fully specified UML models with OCL to tens of pages of written text. Here is what I found that worked best based on most projects that I have been part of:

  1. Static Structure Diagram: this is a class, module, or package diagram. This diagram (though often it is more than one), shows the relationship between parts of the system at a given level of abstraction. The relationships shown include: containment, references, parameters, inheritance, and so on. Record on the diagram indirect and implied dependencies with a note.
  2. Initialization Sequence Diagram: shows how the system is brought up from being on the disk into the operative memory and “ready to transact business” state. This diagram is part of the dynamic model of the system. It shows the order of messages sent or method invocations. Often also shows resources allocated to get up and running.
  3. Shutdown Sequence Diagram: shows how to put back the system or component into the proverbial “bit bucket.” Must complement the initialization sequence diagram and demonstrate that the system is not loosing resources during its operation.
  4. Sequence Diagrams, Activity, and/or State Diagrams for the key “business” of the system or software part. All other activities took place just to get ready to do the business of the system. Now it is time to get the business done. The key sequence diagrams show what the system does, how it provides the value that it is expected to deliver.
  5. Brief Design Commentary to explain the “why” behind key structural and behavioral elements and decisions. This is usually one Wiki page that ties all other documents together into a coherent whole.

Each of these can be created at “every zoom level” in the system. This means that if you are developing a component-based system, then you would have this set of design documents for the system, subsystems, modules, and components. Within each you represent the unique concerns of each software part.

Read Applying UML and Patterns, 3rd. Ed., by Craig Larman for an in-depth course on how to do object-oriented analysis and design on an iterative software development project. As you do design more, review your results, learn from your mistakes, and over time you’ll get better at it.

Resources:

  1. UML basics: An introduction to the Unified Modeling Language, from IBM developerWorks, 2003.
  2. Allen Holub’s UML Quick Reference. 2007.
  3. Book excerpt from OOAD with Applications, by Grady Booch.
  4. UML Modeling in Color, from Wikipedia.

More on Prototyping

To convert the unknowns into knowns, you need to aggressively prototype.

What to Prototype?

Here are some examples:

Prototype the use of an API. Create a prototype to see if the API that you need to use works the way you think it does. This includes confirming that it can be called with parameters of the type and range that you can provide. The API may belong to a component that one of your team mates wrote, or it may belong to a third party library. Either way, you want to be absolutely certain about how to use it. No amount of documentation reading will substitute for a well pointed prototype.

Prototype an algorithm. Create a prototype to prove to yourself beyond reasonable doubt that you can implement the solution to a problem.

Prototype an interaction. Create a prototype to show that the interaction sequence or method that you want to use in your application actually works for the intended use. You can take the prototype and usability test it, incorporate the feedback, and then place it into your application.

What should the prototype look like?

Since we are talking about software development, your prototype should be a software program. A small program. As small as possible. A program that answers one or two critical questions and that’s it.

The prototype should be like the small program that you write to prove to the API writers that their API has a defect. It has to be small and prove one point. Your objective from the prototype to learn something that you didn’t know, or confirm something that you suspected.

How to Prototype?

Pick one or two key questions that you want your prototype to answer and focus your small program answer them. Keep in mind common software engineering and experimentation principles as you decide on how to proceed:

  1. divide & conquer: write multiple prototypes as needed
  2. reduce scope: keep the scope of each prototype to a minimum
  3. isolate concerns: separate the concerns so that you can establish root cause
  4. confirm behavior: confirm that an API behaves exactly the way you had thought that it would under the conditions that you anticipate
  5. validate assumptions: validate that what you think is true, it is indeed so
  6. validate invariants, pre- and post-conditions: half the battle is to identify these; you get extra credit for validating them

I have noticed that after a while some prototypes that started out with the best of intentions, become monsters. Don’t be tempted to stick another one or two topics into an existing prototype. Your prototype will become so complicated that you cannot confirm root causes for outcomes. Then you lost your ability to know things for sure.

Lastly: the most important rule is a well worn one: KISS=Keep It Simple & Sufficient.