CPSC 310 Testing Questions and Answers
Black box tseting - ANSWER-validates programs without any knowledge of how system
is implemented -- relies heavily on predicting problematic inputs by examining public
API signatures and any available documentation of the CUT
Effectiveness - ANSWER-simplest way to reason the effectiveness of a test or test suite
is the measure the probability the test will find a real fault (per unit effort, which can be
something like developer creation/ maintenance time or number of test executions)
end-to-end - ANSWER-aka acceptance testing, typically validate entire customer flows.
They're useful for validating quality attributes (performance and usability) that cannot be
captured in isolation. They're great for ensuring that the system behaves correctly, but
little guidance for failure
Higher/lower testability - ANSWER-Some systems are significantly easier to test than
others due to the way they are constructed -- highly testable system will enable more
effective test for the same cost than a system whose test are largely ineffective
integration - ANSWER-integration tests exercise groups of components to ensure that
their contained units interact correctly together -- touch much larger pieces of the
systems and are more prone to spurious failure. Because they touch many parts of the
code, identity the root of the failure is hard
Repeatability - ANSWER-likelihood that running the same test twice under the same
conditions will yield the same result
smoke/canary - ANSWER-subset of a test suite that typically executes quickly, highly
reliable, high effectiveness. They try to expose a fault as quickly as possible, making it
possible to defer running large swaths of unnecessary tests for a system that is known
to be broken
SUT/CUT - ANSWER-System/ under test // thing that you are actually trying to validate
unit - ANSWER-unit test exercise individual components -- usually functions or
methods, in isolation. It's quick to write and incur low maintenance effort since they
touch such small parts of the system; ensure that the unit fulfills its contract making test
failures more straightforward to understand
White box testing - ANSWER-When testing in a white box manner one typically carefully
examines the program source code in order to identify potentially problematic sets of
inputs or control flow paths
Black box tseting - ANSWER-validates programs without any knowledge of how system
is implemented -- relies heavily on predicting problematic inputs by examining public
API signatures and any available documentation of the CUT
Effectiveness - ANSWER-simplest way to reason the effectiveness of a test or test suite
is the measure the probability the test will find a real fault (per unit effort, which can be
something like developer creation/ maintenance time or number of test executions)
end-to-end - ANSWER-aka acceptance testing, typically validate entire customer flows.
They're useful for validating quality attributes (performance and usability) that cannot be
captured in isolation. They're great for ensuring that the system behaves correctly, but
little guidance for failure
Higher/lower testability - ANSWER-Some systems are significantly easier to test than
others due to the way they are constructed -- highly testable system will enable more
effective test for the same cost than a system whose test are largely ineffective
integration - ANSWER-integration tests exercise groups of components to ensure that
their contained units interact correctly together -- touch much larger pieces of the
systems and are more prone to spurious failure. Because they touch many parts of the
code, identity the root of the failure is hard
Repeatability - ANSWER-likelihood that running the same test twice under the same
conditions will yield the same result
smoke/canary - ANSWER-subset of a test suite that typically executes quickly, highly
reliable, high effectiveness. They try to expose a fault as quickly as possible, making it
possible to defer running large swaths of unnecessary tests for a system that is known
to be broken
SUT/CUT - ANSWER-System/ under test // thing that you are actually trying to validate
unit - ANSWER-unit test exercise individual components -- usually functions or
methods, in isolation. It's quick to write and incur low maintenance effort since they
touch such small parts of the system; ensure that the unit fulfills its contract making test
failures more straightforward to understand
White box testing - ANSWER-When testing in a white box manner one typically carefully
examines the program source code in order to identify potentially problematic sets of
inputs or control flow paths
