Testing: Errors and Test Plans

Testing: Errors and Test Plans

Why testing matters

Testing is a fundamental stage in the software development process. Once a program has been written, it must be tested systematically to verify that it works correctly for all expected inputs — and that it handles unexpected inputs gracefully. A program that works for one input but fails for another is not finished. The SQA N5 specification requires you to understand the types of errors that can occur and how to design a structured test plan to catch them.

Type 1 — Syntax errors

A syntax error occurs when the code breaks the grammar rules of Python. Python reads through your code before running it, and if it finds a syntax error it stops immediately and displays an error message. The program never runs at all.

Common causes of syntax errors include:

• A missing colon after if, elif, else, for, while, or def
• Wrong or missing indentation
• An unclosed bracket, parenthesis, or quotation mark
• A misspelled keyword (e.g. primt instead of print)
• Using = where == is needed inside a condition

Syntax errors are the easiest to fix — Python points directly to the line where the problem was detected. Once fixed, the program will at least start running.

if age >= 16    # SyntaxError: expected ':' at end of if statement
    print("Old enough")

Type 2 — Execution errors (run-time errors)

An execution error (also called a run-time error) occurs after the program has started running. The syntax is correct, so Python has no complaints before the program begins — but something goes wrong during execution and the program crashes with an error message.

Common causes of execution errors include:

• Dividing by zero (ZeroDivisionError)
• Accessing a list index that does not exist (IndexError)
• Trying to convert an incompatible value, such as int("hello") (ValueError)
• Using a variable before it has been assigned a value (NameError)

Execution errors are often caused by unexpected user input. For example, a program that expects a number will crash if the user types a word instead.

scores = [10, 20, 30]
print(scores[5])   # IndexError: list index out of range
                   # (valid indices are 0, 1, 2 — there is no index 5)

Type 3 — Logic errors

A logic error occurs when the program runs from start to finish without crashing, but the output is wrong. Python gives no error message because the code is syntactically valid and executes without incident — the problem is in the programmer's reasoning, not in the code structure.

Logic errors are the hardest to find because there is no error message to guide you. You must trace through the code manually, checking each step, to spot where the logic breaks down.

Common causes of logic errors include:

• Using > when you meant >= (or vice versa) — causes off-by-one issues at boundaries
• Using the wrong operator: + instead of *, or - instead of /
• Initialising a variable to the wrong value (e.g. starting a minimum-finder at 0 instead of array[0])
• A loop running one iteration too many or too few

total = 0
for i in range(1, 6):
    total = total * i   # Logic error: should be total + i
print(total)            # Outputs 0 (not 15) — no error message, just a wrong answer

The trace: total=0. i=1: 0*1=0. i=2: 0*2=0. i=3: 0*3=0. …. Because 0 multiplied by anything is always 0, total never changes. The correct line is total = total + i.

Summary — the three error types at a glance

Types of test data

When testing a program, you must choose test data that covers all scenarios. The SQA specification requires three types:

Normal test data — typical, realistic values well within the valid input range. The program should accept these and produce the correct output. Example: for a program that accepts scores from 0 to 100, normal data might be 45, 67, or 80.

Extreme (boundary) test data — values at the exact edges of the valid range: the minimum and maximum acceptable values. The program should accept these. Extreme data is critical because off-by-one logic errors (using > instead of >=) only reveal themselves at the boundary. Example: for 0 to 100, extreme data is exactly 0 and exactly 100.

Exceptional test data — values outside the valid range, or the wrong data type altogether. A well-written program should reject these and display an appropriate error message. Example: for 0 to 100, exceptional data might be -1, 101, or the text "hello".

Designing a test plan

A test plan is a structured table listing every test case. Writing the test plan before running the program forces you to think about what the correct behaviour should be — the expected result must be written down before testing, not inferred from the output. This is what makes testing rigorous rather than guesswork.

A complete test plan includes for each test case:

• Test number — a reference identifier
• Description — what the test is checking
• Test data — the specific value(s) used
• Expected result — what the program should do with this input
• Type — normal, extreme, or exceptional

For a program that accepts a score between 0 and 100, a test plan might look like this:

Notice that exceptional test data includes values just outside the range (−1, 101) as well as completely wrong data types ("hello"). The values just outside the boundary are particularly important — they test whether the validation uses >= versus > correctly.