Thanassis Tsiodras committed Oct 03, 2017 1 ``````To get 100% decision coverage every decision in the program must take all `````` Thanassis Tsiodras committed Oct 05, 2017 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 ``````possible outcomes at least once. Given that in the code we have two decisions... ```c if (a > 1 && b == 0) { puts("Decision 1 was true"); } else { puts("Decision 1 was false"); } if (a == 2 || x > 1) { puts("Decision 2 was true"); } else { puts("Decision 2 was false"); } ``` ...we need test cases that make those decisions true and false. For example, we `````` Thanassis Tsiodras committed Oct 03, 2017 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 ``````could have the following test cases: a=3, b=0, x=2 → a > 1 && b == 0 (T), a == 2 || x > 1 (T) a=1, b=0, x=1 → a > 1 && b == 0 (F), a == 2 || x > 1 (F) \$ make decision ... ./coverage_demo 3 0 2 Decision 1 was true Decision 2 was true ./coverage_demo 1 0 1 Decision 1 was false Decision 2 was false ... Both decisions are exercised to be both true and false. But notice in the gcov annotated output that all stmts are executed (there are no `####` reported in any source line in the gcov output, we have 100% statement coverage). But there ARE branches that were never executed (because these conditions `````` Thanassis Tsiodras committed Oct 05, 2017 39 40 41 42 43 44 45 46 47 48 49 ``````were never exercised - e.g. `b` was always 0! Look at the last branch counter, i.e. branch 3 of line 10: ```c 2: 10: if (a > 1 && b == 0) { branch 0 taken 1 (fallthrough) branch 1 taken 1 branch 2 taken 1 (fallthrough) branch 3 taken 0 1: 11: puts("Decision 1 was true"); ``` `````` Thanassis Tsiodras committed Oct 03, 2017 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 `````` Now, if we want to get 100% condition coverage, every condition in a decision in the program must take all possible outcomes at least once. In the code we have 4 conditions `a > 1`, `b == 0`, `a == 2`, `x > 1`. We need test cases that make all 4 of the conditions true and false. The previous test cases don’t suffice because the condition `b == 0` is never evaluated to false. That could mean a untested critical scenario that could have a bug. To satisfy condition coverage we could have the following test cases: a = 1, b = 0, x = 2 → a > 1(F), b == 0(T), a == 2(F), x > 1(T) a = 2, b = 1, x = 1 → a > 1(T), b == 0(F), a == 2(T), x > 1(F) \$ make condition ... ./coverage_demo 1 0 2 Decision 1 was false Decision 2 was true ./coverage_demo 2 1 1 Decision 1 was false Decision 2 was true Now each condition has been both true and false, but overall, decision 1 was NEVER true - because its two conditions were once FT, and once TF. `````` Thanassis Tsiodras committed Oct 05, 2017 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 ``````The gcov output indicates this, in that... (a) the stmt coverage is not 100% - notice the `#####`: ```c 2: 10: if (a > 1 && b == 0) { branch 0 taken 1 (fallthrough) branch 1 taken 1 branch 2 taken 0 (fallthrough) branch 3 taken 1 #####: 11: puts("Decision 1 was true"); call 0 never executed -: 12: } else { 2: 13: puts("Decision 1 was false"); ``` (b) ...and the branch coverage is also not 100%, since the FINAL `````` Thanassis Tsiodras committed Oct 03, 2017 91 ``````branch (the one that says in the object code, we've evaluated to true, `````` Thanassis Tsiodras committed Oct 05, 2017 92 93 ``````let's execute the action of the `if`) is also not executed (see above, branch 2 is taken 0 times). `````` Thanassis Tsiodras committed Oct 03, 2017 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 `````` Since both types of coverage are important and one does not guarantee satisfying the other, in practice they are typically combined, and that is called condition decision coverage. For this level of coverage we could have the following test cases: a = 1, b = 1, x = 1 → a > 1(F), b == 0(F), a == 2(F), x > 1(F) a = 2, b = 0, x = 2 → a > 1(T), b == 0(T), a == 2(T), x > 1(T) \$ make conditiondecision ... ./coverage_demo 1 1 1 Decision 1 was false Decision 2 was false ./coverage_demo 2 0 2 Decision 1 was true Decision 2 was true Now, notice that although these cases would provide both levels of coverage some conditions are being masked and therefore never evaluated. gcov indicates this, since there are branches that are still, never executed. For example, `a = 1` masks the condition `b == 0` in the decision `(a > 1 && b == 0)` because it makes the condition `a > 1` false and therefore the executable doesn't need to evaluate the other condition to determine the outcome of the `````` Thanassis Tsiodras committed Oct 05, 2017 120 121 122 123 124 125 126 ``````AND expression. These are the 'short-circuit' semantics of the C language. Hence, we need to modify the test cases to ensure the coverage of masking conditions, and that's called modified condition decision coverage (MCDC): every condition in a decision has been shown to independently affect that decision’s outcome. The test cases below satisfy MCDC coverage and therefore also satisfy decision coverage and condition coverage: `````` Thanassis Tsiodras committed Oct 03, 2017 127 128 129 130 131 132 133 `````` a = 2, b = 0, x = 1 → a > 1 && b == 0 (T), a == 2 || x > 1 (T) a = 3, b = 1, x = 2 → a > 1 && b == 0 (F), a == 2 || x > 1 (T) a = 1, b = 0, x = 1 → a > 1 && b == 0 (F), a == 2 || x > 1 (F) \$ make mcdc `````` Thanassis Tsiodras committed Oct 05, 2017 134 135 136 137 138 139 140 141 142 143 144 145 ``````At which point, all statements are executed, and all branches are reported as executed by gcov: \$ make mcdc | grep branch branch 0 taken 2 (fallthrough) branch 1 taken 1 branch 2 taken 1 (fallthrough) branch 3 taken 1 branch 0 taken 2 (fallthrough) branch 1 taken 1 branch 2 taken 1 (fallthrough) branch 3 taken 1 `````` Thanassis Tsiodras committed Oct 03, 2017 146 `````` `````` Thanassis Tsiodras committed Oct 05, 2017 147 ``*The content above is based on [a nice (but slightly buggy) Quora article](https://www.quora.com/What-is-the-difference-between-decision-coverage-and-condition-coverage-when-it-comes-to-code-coverage).*``