CITS5501 lab 5 (week 6) – version control, logic-based testing

1. Project version control

The week 5 lectures include a workshop on using Git and GitHub for version control.

If this is the first time you have used Git and GitHub, you should work through the resources included there, including the Github Hello World Step by Step tutorial.

If you have used Git and GitHub before, you might like to think in more depth about how version control can contribute to (or pose problems for) software quality. The questions below ask you to reflect on this.

Question

What is the basic purpose of version control? How does it differ from simply keeping copies of files with different names or dates?

Question

How might version control make software development easier or harder? You may want to check online resources to help you answer these questions.

• How can version control make it easier to trace the source of a bug?
• In what ways might poor version control practices (e.g. unclear commit messages, cluttered history) make testing or debugging harder?
• How might including large generated files in a repository affect development speed and reliability?
   

Question

In group assessments from previous units, has your group adopted any conventions or best practices when using version control? If so, do you think they were useful – and in what ways? If not, did you encounter any problems where adopting a convention might have made things easier?

Think about the following:

• Why might a team agree on a commit message convention or branching strategy?
• What problems can arise if different team members ignore these agreements?
• How could poor version control practices create extra work for testers or integrators?
   

Question

Can you think of any indicators in a version control repository that might be a sign of good version control discipline (or conversely, or poor version control discipline)? If you were reviewing a peer’s repository, what would you look for as signs of good or bad version control discipline?

Version control tools can either support software quality or undermine it, depending on how they’re used. Good practices make repositories easier to work with, simplify testing, and reduce the risk of errors. But poor practices can lead to confusion, wasted time, and hidden bugs.

In a well-managed repository, the version control history should clearly show

• what changed
• when it changed
• and why.

Information about the first two items comes directly from the version control tool – all version control tools will show what content in what files was changed, and when a commit occurred. However, providing reasons why the change was made can only be done by the developer who committed it. In version control systems, commit messages are used to explain the reasons for a change. You can read more about what makes for good or bad commit messages in Simon Tatham’s guide to “Writing commit messages”.

When used properly, version control systems make it much easier to solve problems like

• Diagnosing when and how a bug was introduced.
• Understanding the reasons for implementation decisions.
• Reverting buggy software to a known good state.
• Letting teams work on multiple features in parallel (using branches and merges).

Version control best practices

We won’t discuss version control best practices in exhaustive detail here, but will highlight a few important ones. For more detail, you can read Michael Ernst’s “Version control concepts and best practices”.

• Avoid committing generated files. They add “noise” to the version control history, are a common source of merge conflicts, and can quickly bloat the repository.
• Avoid versioning binary files where possible. Instead, version the text format or build recipe. If you do need to track binary files, consider specialised tools (e.g. Git LFS).
• Write informative commit messages. A good message explains what changed and why. If your team follows a standard (e.g. Conventional Commits), stick to it.
• Commit early, commit often. Don’t leave work uncommitted for any extended period of time – you lose the advantage of having a version control system if you do that.
• Create commits at logical points. Each commit should represent a coherent change, small enough to review and test in isolation.
• Review before merging. Code review not only catches bugs, it also enforces quality in commit history and documentation.

You might notice that some of these practices are in tension – if you commit early and commit often to stop work from being lost, how can you also ensure that you’ve provided good commit messages, or created commits at logical points?

The answer to this apparent paradox is that it’s possible to rewrite the local history of your repository, and in fact it’s a good idea to

• Tidy history before sharing. Squash or reorganise commits so that the branch history tells a clear story.

Once you’ve pushed your commits to a remote repository, it’s generally a bad idea to alter them, since other team-mates’ work may be affected by any changes. But before you push your commits, it’s perfectly alright – in fact, recommended – to tidy up the history before pushing it. Read Carlos Schults’s post “Make Your Git History Look Beautiful Using Amend and Rebase” for more information on how to do this.

2. Logic notation

The remaining sections of this lab focus on assessing whether a test suite thoroughly exercises (i.e., has good coverage of) the logic expressions contained in the component under test.

When writing logic expressions, we will often use mathematical notation for “and”, “or”, and “not”:

• \(\wedge\) – “and”
• \(\vee\) – “or”
• \(\neg\) – “not”

This notation is independent of any language; it could be turned into Java, or C, or Python – each of which uses different logical operators – depending on what language our system and our tests are implemented in.

If writing actual Java code, however, we use the normal Java logical operators:

• && – “and”
• || – “or”
• ! – “not”

3. Terminology – clauses and predicates

If you need to, review the lecture material and recommended readings that explain what predicates and clauses are.

What are the clauses in the predicates below?

1. \(((f \leqslant g) \wedge (x > 0)) \vee (M \wedge (e < d +c))\)

2. \(G \vee ((m > a) \vee (s \leqslant o + n)) \wedge U\)

4. Making clauses active

To make a particular clause \(c\) in some predicate active means to assign values to variables so that the truth-value of the whole predicate depends on \(c\).

When coming up with test values which make clauses active, the easiest way of showing your test values is in a table.

E.g. Suppose we have a predicate \(s \wedge (m \vee w)\), where

• \(m\) = “the moon is full”
• \(s\) = “the sky is clear”
• \(w\) = “the wind is calm”

If asked to come up with test inputs which make each clause active in turn, and achieve Restricted Active Clause Coverage, we could show them like this:

(Here, we aren’t told what the expected outcome is if the predicate comes out true or false; if we were, we could add a column “Expected outcome” which listed this.)

If you aren’t told the exact types of variables or methods used in a predicate, that means you should be able to work them out from context. For example, for the predicate

\[ (x > 0) \vee (M \wedge (e < d +c)) \]

you can assume that \(M\) is a boolean, and that \(x\), \(c\), \(d\) and \(e\) are some numeric type (such as int).

For each of the clauses in the predicates below, identify test inputs which will make the clause active (that is: state what values need to be assigned to the variables in the predicate), and vary that clause so it takes on both true and false values. (In other words: write test values that achieve Restricted Active Clause Coverage.) Explain your reasoning.

1. \(A \vee (B \wedge \neg C)\)
2. \(x > 0 \; \vee (M \wedge (e < d +c))\)
3. \(G \vee (m \geqslant a) \vee H \wedge U\)

5. Scenario – trap-doors

Suppose a component under test has the following requirements:

Represent the component as a set of logic expressions. You should explain what each variable in your expressions means. (For an example, look in section 2 at the way we gave definitions for the variables in the predicate \(s \wedge (m \vee w)\).)

(Hint: if you’re stuck, try writing out what the component does as one or more “if” statements, in pseudocode. Then recall that the set of all predicates in a system means the set of all logical expressions found in things like “if” statements.)

6. Scenario – login page

Suppose you are part of a team developing a website called “RateMyVeterinarian”, where people can log in and provide anonymous reviews of the veterinarian services they use.

Requirements for the site are currently being finalised, and one requirement is stated as follows:

1. How easy to understand do you think this requirement is? If you think it could be made easier to understand, suggest how.

2. One of your colleagues suggests that because correctly authenticating users (and keeping their details secure) is an important feature, this requirement should be thoroughly tested – so you should design a test suite that meets RAC (Restricted Active Clause) levels of coverage. Do you agree? Why or why not?

7. Tips and tricks

When solving problems that involve logic-based testing, make sure you understand the difference between expressions and statements in the programming language or languages that you’re using.

For instance, simple Java if statements are of the form:

   if ( Boolean expression ) { Statements }

Statements are parts of the language that do things; we usually execute them in order to achieve some side effect. Boolean expressions evaluate to a Java boolean value, and we typically intend that they should not have side effects – they just evaluate to true or false.

When we do logic-based testing, we’re focusing on the collection of logic expressions of a program or system that control program flow, and working out how to write tests that thoroughly exercise the component parts (the clauses) of those expressions.

Confusing expressions with statements, or vice versa, is a common reason for students to do poorly in test or exam questions involving logic-based testing.