Week 5 [Sep 9] - Topics

A software requirement specifies a need to be fulfilled by the software product.

A software project may be,

• a brown-field project i.e., develop a product to replace/update an existing software product
• a green-field project i.e., develop a totally new system with no precedent

In either case, requirements need to be gathered, analyzed, specified, and managed.

Requirements come from stakeholders.

Identifying requirements is often not easy. For example, stakeholders may not be aware of their precise needs, may not know how to communicate their requirements correctly, may not be willing to spend effort in identifying requirements, etc.

Requirements can be divided into two in the following way:

1. Functional requirements specify what the system should do.
2. Non-functional requirements specify the constraints under which system is developed and operated.

We may have to spend an extra effort in digging NFRs out as early as possible because,

1. NFRs are easier to miss e.g., stakeholders tend to think of functional requirements first
2. sometimes NFRs are critical to the success of the software. E.g. A web application that is too slow or that has low security is unlikely to succeed even if it has all the right functionality.

Here are some characteristics of well-defined requirements ^{[📖 zielczynski]}:

• Unambiguous
• Testable (verifiable)
• Clear (concise, terse, simple, precise)
• Correct
• Understandable
• Feasible (realistic, possible)
• Independent
• Atomic
• Necessary
• Implementation-free (i.e. abstract)

Besides these criteria for individual requirements, the set of requirements as a whole should be

• Consistent
• Non-redundant
• Complete

Requirements can be prioritized based the importance and urgency, while keeping in mind the constraints of schedule, budget, staff resources, quality goals, and other constraints.

A common approach is to group requirements into priority categories. Note that all such scales are subjective, and stakeholders define the meaning of each level in the scale for the project at hand.

Some requirements can be discarded if they are considered ‘out of scope’.

In a brainstorming session there are no "bad" ideas. The aim is to generate ideas; not to validate them. Brainstorming encourages you to "think outside the box" and put "crazy" ideas on the table without fear of rejection.

Studying existing products can unearth shortcomings of existing solutions that can be addressed by a new product. Product manuals and other forms of documentation of an existing system can tell us how the existing solutions work.

Observing users in their natural work environment can uncover product requirements. Usage data of an existing system can also be used to gather information about how an existing system is being used, which can help in building a better replacement e.g. to find the situations where the user makes mistakes when using the current system.

Surveys can be used to solicit responses and opinions from a large number of stakeholders regarding a current product or a new product.

Interviewing stakeholders and domain experts can produce useful information that project requirements.

Focus groups are a kind of informal interview within an interactive group setting. A group of people (e.g. potential users, beta testers) are asked about their understanding of a specific issue, process, product, advertisement, etc.

Prototyping can uncover requirements, in particular, those related to how users interact with the system. UI prototypes or mock ups are often used in brainstorming sessions, or in meetings with the users to get quick feedback from them.

Prototyping can be used for discovering as well as specifying requirements e.g. a UI prototype can serve as a specification of what to build.

A textual description (i.e. prose) can be used to describe requirements. Prose is especially useful when describing abstract ideas such as the vision of a product.

Avoid using lengthy prose to describe requirements; they can be hard to follow.

A common format for writing user stories is:

We can write user stories on index cards or sticky notes, and arrange on walls or tables, to facilitate planning and discussion. Alternatively, we can use a software (e.g., GitHub Project Boards, Trello, Google Docs, ...) to manage user stories digitally.

^{[credit: https://www.flickr.com/photos/jakuza/2682466984/]}

^{[credit: https://www.flickr.com/photos/jakuza/with/2726048607/]}

^{[credit: https://commons.wikimedia.org/wiki/File:User_Story_Map_in_Action.png]}

The {benefit} can be omitted if it is obvious.

It is recommended to confirm there is a concrete benefit even if you omit it from the user story. If not, you could end up adding features that have no real benefit.

You can add more characteristics to the {user role} to provide more context to the user story.

You can write user stories at various levels. High-level user stories, called epics (or themes) cover bigger functionality. You can then break down these epics to multiple user stories of normal size.

You can add conditions of satisfaction to a user story to specify things that need to be true for the user story implementation to be accepted as ‘done’.

Other useful info that can be added to a user story includes (but not limited to)

• Priority: how important the user story is
• Size: the estimated effort to implement the user story
• Urgency: how soon the feature is needed

User stories capture user requirements in a way that is convenient for scoping, estimation, and scheduling.

[User stories] strongly shift the focus from writing about features to discussing them. In fact, these discussions are more important than whatever text is written. _{[Mike Cohn, MountainGoat Software 🔗]}

User stories differ from traditional requirements specifications mainly in the level of detail. User stories should only provide enough details to make a reasonably low risk estimate of how long the user story will take to implement. When the time comes to implement the user story, the developers will meet with the customer face-to-face to work out a more detailed description of the requirements. [more...]

User stories can capture non-functional requirements too because even NFRs must benefit some stakeholder.

While use cases can be recorded on physical paper in the initial stages, an online tool is more suitable for longer-term management of user stories, especially if the team is not co-located.

A supplementary requirements section can be used to capture requirements that do not fit elsewhere. Typically, this is where most Non Functional Requirements will be listed.

Programs should be written and polished until they acquire publication quality. _{--Niklaus Wirth}

Among various dimensions of code quality, such as run-time efficiency, security, and robustness, one of the most important is understandability. This is because in any non-trivial software project, code needs to be read, understood, and modified by other developers later on. Even if we do not intend to pass the code to someone else, code quality is still important because we all become 'strangers' to our own code someday.

int subsidy() {
    int subsidy;
    if (!age) {
        if (!sub) {
            if (!notFullTime) {
                subsidy = 500;
            } else {
                subsidy = 250;
            }
        } else {
            subsidy = 250;
        }
    } else {
        subsidy = -1;
    }
    return subsidy;
}

int calculateSubsidy() {
    int subsidy;
    if (isSenior) {
        subsidy = REJECT_SENIOR;
    } else if (isAlreadySubsidised) {
        subsidy = SUBSIDISED_SUBSIDY;
    } else if (isPartTime) {
        subsidy = FULLTIME_SUBSIDY * RATIO;
    } else {
        subsidy = FULLTIME_SUBSIDY;
    }
    return subsidy;
}

def calculate_subs():
    if not age:
        if not sub:
            if not not_fulltime:
                subsidy = 500
            else:
                subsidy = 250
        else:
            subsidy = 250
    else:
        subsidy = -1
    return subsidy

def calculate_subsidy():
    if is_senior:
        return REJECT_SENIOR
    elif is_already_subsidised:
        return SUBSIDISED_SUBSIDY
    elif is_parttime:
        return FULLTIME_SUBSIDY * RATIO
    else:
        return FULLTIME_SUBSIDY

Be wary when a method is longer than the computer screen, and take corrective action when it goes beyond 30 LOC (lines of code). The bigger the haystack, the harder it is to find a needle.

If you need more than 3 levels of indentation, you're screwed anyway, and should fix your program. _{--Linux 1.3.53 CodingStyle}

In particular, avoid arrowhead style code.

int subsidy() {
    int subsidy;
    if (!age) {
        if (!sub) {
            if (!notFullTime) {
                subsidy = 500;
            } else {
                subsidy = 250;
            }
        } else {
            subsidy = 250;
        }
    } else {
        subsidy = -1;
    }
    return subsidy;
}

int calculateSubsidy() {
    int subsidy;
    if (isSenior) {
        subsidy = REJECT_SENIOR;
    } else if (isAlreadySubsidised) {
        subsidy = SUBSIDISED_SUBSIDY;
    } else if (isPartTime) {
        subsidy = FULLTIME_SUBSIDY * RATIO;
    } else {
        subsidy = FULLTIME_SUBSIDY;
    }
    return subsidy;
}

def calculate_subs():
    if not age:
        if not sub:
            if not not_fulltime:
                subsidy = 500
            else:
                subsidy = 250
        else:
            subsidy = 250
    else:
        subsidy = -1
    return subsidy

def calculate_subsidy():
    if is_senior:
        return REJECT_SENIOR
    elif is_already_subsidised:
        return SUBSIDISED_SUBSIDY
    elif is_parttime:
        return FULLTIME_SUBSIDY * RATIO
    else:
        return FULLTIME_SUBSIDY

Avoid complicated expressions, especially those having many negations and nested parentheses. If you must evaluate complicated expressions, have it done in steps (i.e. calculate some intermediate values first and use them to calculate the final value).

return ((length < MAX_LENGTH) || (previousSize != length)) && (typeCode == URGENT);

boolean isWithinSizeLimit = length < MAX_LENGTH;
boolean isSameSize = previousSize != length;
boolean isValidCode = isWithinSizeLimit || isSameSize;

boolean isUrgent = typeCode == URGENT;

return isValidCode && isUrgent;

return ((length < MAX_LENGTH) or (previous_size != length)) and (type_code == URGENT)

is_within_size_limit = length < MAX_LENGTH
is_same_size = previous_size != length
is_valid_code = is_within_size_limit or is_same_size

is_urgent = type_code == URGENT

return is_valid_code and is_urgent

The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague. _{-- Edsger Dijkstra}

When the code has a number that does not explain the meaning of the number, we call that a "magic number" (as in "the number appears as if by magic"). Using a named constant makes the code easier to understand because the name tells us more about the meaning of the number.

return 3.14236;
...
return 9;

static final double PI = 3.14236;
static final int MAX_SIZE = 10;
...
return PI;
...
return MAX_SIZE-1;

return 3.14236
...
return 9

PI = 3.14236
MAX_SIZE = 10
...
return PI
...
return MAX_SIZE-1

Similarly, we can have ‘magic’ values of other data types.

"Error 1432"  // A magic string!

In general, try to avoid any magic literals.

Make the code as explicit as possible, even if the language syntax allows them to be implicit. Here are some examples:

• [Java] Use explicit type conversion instead of implicit type conversion.
• [Java, Python] Use parentheses/braces to show grouping even when they can be skipped.
• [Java, Python] Use enumerations when a certain variable can take only a small number of finite values. For example, instead of declaring the variable 'state' as an integer and using values 0,1,2 to denote the states 'starting', 'enabled', and 'disabled' respectively, declare 'state' as type SystemState and define an enumeration SystemState that has values 'STARTING', 'ENABLED', and 'DISABLED'.

Lay out the code so that it adheres to the logical structure. The code should read like a story. Just like we use section breaks, chapters and paragraphs to organize a story, use classes, methods, indentation and line spacing in your code to group related segments of the code. For example, you can use blank lines to group related statements together. Sometimes, the correctness of your code does not depend on the order in which you perform certain intermediary steps. Nevertheless, this order may affect the clarity of the story you are trying to tell. Choose the order that makes the story most readable.

Avoid things that would make the reader go ‘huh?’, such as,

• unused parameters in the method signature
• similar things that look different
• different things that look similar
• multiple statements in the same line
• data flow anomalies such as, pre-assigning values to variables and modifying it without any use of the pre-assigned value

As the old adage goes, "keep it simple, stupid” (KISS). Do not try to write ‘clever’ code. For example, do not dismiss the brute-force yet simple solution in favor of a complicated one because of some ‘supposed benefits’ such as 'better reusability' unless you have a strong justification.

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. _{--Brian W. Kernighan}

Programs must be written for people to read, and only incidentally for machines to execute. _{--Abelson and Sussman}

Optimizing code prematurely has several drawbacks:

• We may not know which parts are the real performance bottlenecks. This is especially the case when the code undergoes transformations (e.g. compiling, minifying, transpiling, etc.) before it becomes an executable. Ideally, you should use a profiler tool to identify the actual bottlenecks of the code first, and optimize only those parts.
• Optimizing can complicate the code, affecting correctness and understandability
• Hand-optimized code can be harder for the compiler to optimize (the simpler the code, the easier for the compiler to optimize it). In many cases a compiler can do a better job of optimizing the runtime code if you don't get in the way by trying to hand-optimize the source code.

A popular saying in the industry is make it work, make it right, make it fast which means in most cases getting the code to perform correctly should take priority over optimizing it. If the code doesn't work correctly, it has no value on matter how fast/efficient it it.

Premature optimization is the root of all evil in programming. _{--Donald Knuth}

Note that there are cases where optimizing takes priority over other things e.g. when writing code for resource-constrained environments. This guideline simply a caution that you should optimize only when it is really needed.

Avoid varying the level of abstraction within a code fragment. Note: The Productive Programmer (by Neal Ford) calls this the SLAP principle i.e. Single Level of Abstraction Per method.

readData();
salary = basic*rise+1000;
tax = (taxable?salary*0.07:0);
displayResult();

readData();
processData();
displayResult();

The happy path (i.e. the execution path taken when everything goes well) should be clear and prominent in your code. Restructure the code to make the happy path unindented as much as possible. It is the ‘unusual’ cases that should be indented. Someone reading the code should not get distracted by alternative paths taken when error conditions happen. One technique that could help in this regard is the use of guard clauses.

if (!isUnusualCase) {  //detecting an unusual condition
    if (!isErrorCase) {
        start();    //main path
        process();
        cleanup();
        exit();
    } else {
        handleError();
    }
} else {
    handleUnusualCase(); //handling that unusual condition
}

if (isUnusualCase) { //Guard Clause
    handleUnusualCase();
    return;
}

if (isErrorCase) { //Guard Clause
    handleError();
    return;
}

start();
process();
cleanup();
exit();

Proper naming improves the readability. It also reduces bugs caused by ambiguities regarding the intent of a variable or a method.

There are only two hard things in Computer Science: cache invalidation and naming things. _{-- Phil Karlton}

Every system is built from a domain-specific language designed by the programmers to describe that system. Functions are the verbs of that language, and classes are the nouns. _{― Robert C. Martin, Clean Code: A Handbook of Agile Software Craftsmanship}

Use nouns for classes/variables and verbs for methods/functions.

Distinguish clearly between single-valued and multivalued variables.

Person student;
ArrayList<Person> students;

name = 'Jim'
names = ['Jim', 'Alice']

Use correct spelling in names. Avoid 'texting-style' spelling. Avoid foreign language words, slang, and names that are only meaningful within specific contexts/times e.g. terms from private jokes, a TV show currently popular in your country

A name is not just for differentiation; it should explain the named entity to the reader accurately and at a sufficient level of detail.

If the name has multiple words, they should be in a sensible order.

Imagine going to the doctor's and saying "My eye1 is swollen"! Don’t use numbers or case to distinguish names.

While it is preferable not to have lengthy names, names that are 'too short' are even worse. If you must abbreviate or use acronyms, do it consistently. Explain their full meaning at an obvious location.

Related things should be named similarly, while unrelated things should NOT.

Avoid misleading or ambiguous names (e.g. those with multiple meanings), similar sounding names, hard-to-pronounce ones (e.g. avoid ambiguities like "is that a lowercase L, capital I or number 1?", or "is that number 0 or letter O?"), almost similar names.

It is safer to use language constructs in the way they are meant to be used, even if the language allows shortcuts. Some such coding practices are common sources of bugs. Know them and avoid them.

Always include a default branch in case statements.

Furthermore, use it for the intended default action and not just to execute the last option. If there is no default action, you can use the 'default' branch to detect errors (i.e. if execution reached the default branch, throw an exception). This also applies to the final else of an if-else construct. That is, the final else should mean 'everything else', and not the final option. Do not use else when an if condition can be explicitly specified, unless there is absolutely no other possibility.

if (red) print "red";
else print "blue";

if (red) print "red";
else if (blue) print "blue";
else error("incorrect input");

• Use one variable for one purpose. Do not reuse a variable for a different purpose other than its intended one, just because the data type is the same.
• Do not reuse formal parameters as local variables inside the method.

double computeRectangleArea(double length, double width) {
    length = length * width;
    return length;
}

double computeRectangleArea(double length, double width) {
    double area;
    area = length * width;
    return area;
}

Never write an empty catch statement. At least give a comment to explain why the catch block is left empty.

We all feel reluctant to delete code we have painstakingly written, even if we have no use for that code any more ("I spent a lot of time writing that code; what if we need it again?"). Consider all code as baggage you have to carry; get rid of unused code the moment it becomes redundant. If you need that code again, simply recover it from the revision control tool you are using. Deleting code you wrote previously is a sign that you are improving.

Minimize global variables. Global variables may be the most convenient way to pass information around, but they do create implicit links between code segments that use the global variable. Avoid them as much as possible.

Define variables in the least possible scope. For example, if the variable is used only within the if block of the conditional statement, it should be declared inside that if block.

The most powerful technique for minimizing the scope of a local variable is to declare it where it is first used. _{-- Effective Java, by Joshua Bloch}

Resources:

• Refactoring: Reduce Scope of Variable

Code duplication, especially when you copy-paste-modify code, often indicates a poor quality implementation. While it may not be possible to have zero duplication, always think twice before duplicating code; most often there is a better alternative.

This guideline is closely related to the DRY Principle .

Good code is its own best documentation. As you’re about to add a comment, ask yourself, ‘How can I improve the code so that this comment isn’t needed?’ Improve the code and then document it to make it even clearer. _{--Steve McConnell, Author of Clean Code}

Some think commenting heavily increases the 'code quality'. This is not so. Avoid writing comments to explain bad code. Improve the code to make it self-explanatory.

If the code is self-explanatory, refrain from repeating the description in a comment just for the sake of 'good documentation'.

// increment x
x++;

//trim the input
trimInput();

// increment x
x = x + 1

//trim the input
trim_input()

Do not write comments as if they are private notes to yourself. Instead, write them well enough to be understood by another programmer. One type of comments that is almost always useful is the header comment that you write for a class or an operation to explain its purpose.

// a quick trim function used to fix bug I detected overnight
void trimInput(){
    ....
}

/** Trims the input of leading and trailing spaces */
void trimInput(){
    ....
}

# a quick trim function used to fix bug I detected overnight
def trim_input():
    ...

def trim_input():
"""Trim the input of leading and trailing spaces"""
    ...

Comments should explain what and why aspect of the code, rather than the how aspect.

What : The specification of what the code supposed to do. The reader can compare such comments to the implementation to verify if the implementation is correct

/** Removes all spaces from the {@code input} */
void compact(String input){
    input.trim();
}

Why : The rationale for the current implementation.

// Remove spaces to comply with IE23.5 formatting rules
compact(input);

How : The explanation for how the code works. This should already be apparent from the code, if the code is self-explanatory. Adding comments to explain the same thing is redundant.

// return true if both left end and right end are correct or the size has not incremented
return (left && right) || (input.size() == size);

boolean isSameSize = (input.size() == size) ;
return (isLeftEndCorrect && isRightEndCorrect) || isSameSize;

The first version of the code you write may not be of production quality. It is OK to first concentrate on making the code work, rather than worry over the quality of the code, as long as you improve the quality later. This process of improving a program's internal structure in small steps without modifying its external behavior is called refactoring.

• Refactoring is not rewriting: Discarding poorly-written code entirely and re-writing it from scratch is not refactoring because refactoring needs to be done in small steps.
• Refactoring is not bug fixing: By definition, refactoring is different from bug fixing or any other modifications that alter the external behavior (e.g. adding a feature) of the component in concern.

Given below are two common refactorings ( more).

if (isSpecialDeal()) {
    total = price * 0.95;
    send();
} else {
    total = price * 0.98;
    send();
}

if (isSpecialDeal()){
    total = price * 0.95;
} else {
    total = price * 0.98;
}
send();

if is_special_deal:
    total = price * 0.95
    send()
else:
    total = price * 0.98
    send()

if is_special_deal:
    total = price * 0.95
else:
    total = price * 0.98
    
send()

void printOwing() {
    printBanner();

    //print details
    System.out.println("name:	" + name);
    System.out.println("amount	" + getOutstanding());
}

void printOwing() {
    printBanner();
    printDetails(getOutstanding());
}

void printDetails (double outstanding) {
    System.out.println("name:	" + name);
    System.out.println("amount	" + outstanding);
}

def print_owing():
    print_banner()

    //print details
    print("name:	" + name)
    print("amount	" + get_outstanding())

def print_owing():
    print_banner()
    print_details(get_outstanding())

def print_details(amount):
    print("name:	" + name)
    print("amount	" + amount)

Some IDEs have built in support for basic refactorings such as automatically renaming a variable/method/class in all places it has been used.

Refactoring, even if done with the aid of an IDE, may still result in regressions. Therefore, each small refactoring should be followed by regression testing.

• Introduction to Refactoring (in Intellij IDEA) : An article on refactorings available in Intellij IDEA.

Given below are some more commonly used refactorings. A more comprehensive list is available at refactoring-catalog .

1. Consolidate Conditional Expression
2. Decompose Conditional
3. Inline Method
4. Remove Double Negative
5. Replace Magic Literal
6. Replace Nested Conditional with Guard Clauses
7. Replace Parameter with Explicit Methods
8. Reverse Conditional
9. Split Loop
10. Split Temporary Variable

We know that it is important to refactor frequently so as to avoid the accumulation of ‘messy’ code which might get out of control. But how much refactoring is too much refactoring? It is too much refactoring when the benefits no longer justify the cost. The costs and the benefits depend on the context. That is why some refactorings are ‘opposites’ of each other (e.g. extract method vs inline method).

Assertions are used to define assumptions about the program state so that the runtime can verify them. An assertion failure indicates a possible bug in the code because the code has resulted in a program state that violates an assumption about how the code should behave.

If the runtime detects an assertion failure, it typically take some drastic action such as terminating the execution with an error message. This is because an assertion failure indicates a possible bug and the sooner the execution stops, the safer it is.

int timeout = Config.getTimeout(); 

Use the assert keyword to define assertions.

x = getX();
assert x == 0 : "x should be 0";
...

Assertions can be disabled without modifying the code.

Enable assertions in Intellij (how?) and get an assertion to fail temporarily (e.g. insert an assert false into the code temporarily) to confirm assertions are being verified.

Java assert vs JUnit assertions: They are similar in purpose but JUnit assertions are more powerful and customized for testing. In addition, JUnit assertions are not disabled by default. We recommend you use JUnit assertions in test code and Java assert in functional code.

It is recommended that assertions be used liberally in the code. Their impact on performance is considered low and worth the additional safety they provide.

Do not use assertions to do work because assertions can be disabled. If not, your program will stop working when assertions are not enabled.

...
assert writeFile() : "File writing is supposed to return true";

Assertions are suitable for verifying assumptions about Internal Invariants, Control-Flow Invariants, Preconditions, Postconditions, and Class Invariants. Refer to [Programming with Assertions (second half)] to learn more.

Exceptions and assertions are two complementary ways of handling errors in software but they serve different purposes. Therefore, both assertions and exceptions should be used in code.

• The raising of an exception indicates an unusual condition created by the user (e.g. user inputs an unacceptable input) or the environment (e.g., a file needed for the program is missing).
• An assertion failure indicates the programmer made a mistake in the code (e.g., a null value is returned from a method that is not supposed to return null under any circumstances).

Java 8 introduced a number of new features (e.g. Lambdas, Streams) that are not trivial to learn but also extremely useful to know.

Here is an overview of new Java 8 features .(written by Benjamin Winterberg)

Combining parts of a software product to form a whole is called integration. It is also one of the most troublesome tasks and it rarely goes smoothly.

Build automation tools automate the steps of the build process, usually by means of build scripts.

In a non-trivial project, building a product from source code can be a complex multi-step process. For example, it can include steps such as to pull code from the revision control system, compile, link, run automated tests, automatically update release documents (e.g. build number), package into a distributable, push to repo, deploy to a server, delete temporary files created during building/testing, email developers of the new build, and so on. Furthermore, this build process can be done ‘on demand’, it can be scheduled (e.g. every day at midnight) or it can be triggered by various events (e.g. triggered by a code push to the revision control system).

Some of these build steps such as to compile, link and package are already automated in most modern IDEs. For example, several steps happen automatically when the ‘build’ button of the IDE is clicked. Some IDEs even allow customization to this build process to some extent.

However, most big projects use specialized build tools to automate complex build processes.

Some build tools also serve as dependency management tools. Modern software projects often depend on third party libraries that evolve constantly. That means developers need to download the correct version of the required libraries and update them regularly. Therefore, dependency management is an important part of build automation. Dependency Management tools can automate that aspect of a project.

An extreme application of build automation is called continuous integration (CI) in which integration, building, and testing happens automatically after each code change.

A natural extension of CI is Continuous Deployment (CD) where the changes are not only integrated continuously, but also deployed to end-users at the same time.