SPCE 630
What are the 3 indicators of "trustworthy" measurement? - ANS-Validity, accuracy, and
reliability.
Describe validity in terms of one of the 3 indicators of "trustworthy" measurement. What
does it measure? - ANS-Requires 3 equally important elements: (a) measuring directly
a socially significant target behavior, (b) measuring a dimension of the target behavior
relevant to the question or concern about the behavior and (c) ensuring that the data
are representative of the behavior's occurrence under conditions and during times that
are most relevant to the question or concern about the behavior.
Describe accuracy in terms of one of the 3 indicators of "trustworthy" measurement.
What does it measure? - ANS-Refers to the extent to which the observed value
matches the true state or true value of the event as it exists in nature. It must be
determined if there is a measurement bias.
Describe reliability in terms of one of the 3 indicators of "trustworthy" measurement.
What does it measure? - ANS-When measuring a procedure yields the same outcome
each time. It measures consistency of data collection.
How do we ensure our data are valid, accurate, and reliable? - ANS-Researchers and
practitioners who asses the accuracy of their data can (a) determine early in an analysis
whether the data re usable for making experimental or treatment decisions, (b) discover
and correct measurement errors, (c) detect consistent patterns of measurement error
that can lead to the overall improvement or calibration of the measurement system, and
(d) communicate to others the relative trustworthiness of the data.
What are some procedures that provide direct and indirect measurement? - ANS-Direct
observation is an example of direct measurement and questionnaires filled out by
parents is an example of indirect measurement.
Describe the difference between continuous recording and sampling recording. Give
examples of each, and advantages and disadvantages. How do we do each type of
recording? 106-107 - ANS-Continuous recording is considered the gold standard in
applied behavior analysis as it continuously records behaviors in a session. sampling
recording differs as data is recorded in samples or increments that have been
,predetermined. Sample recording may be utilized when an observer is collecting data
for multiple students at one time.
Name the three threats to measurement accuracy and reliability. - ANS-Human error is
the biggest threat to measurement accuracy and reliability. Factors that contribute to
human measurement error include poorly designed measurement systems, inadequate
observer training and expectations about what the data should look like.
poorly designed measurement systems - ANS-An example would be an unnecessarily
cumbersome and difficult to use measurement system which can create a needless loss
of accuracy and reliability.
inadequate observer training - ANS-observers who are not trained enough. Explicit and
systematic training of observers i essential for the collection of trustworthy data.
unintended influences on observers - ANS-Presuppositions an observer may hold about
the expected outcomes of the data and an observer's awareness that others are
measuring the same behavior.
How do we maximize the quality of observers who are taking our data? 108-109 -
ANS-Provide a systematic approach that covers methods such as sample vignettes,
narrative descriptions, video sequences, role playing and practice sessions int eh
environment
Describe how we assess the accuracy and reliability of data, including the differences
between obtained and true values of data. Describe how, if you can, look at data and
know whether those data are obtained or true or both? 110-11 - ANS-calculate the
correspondence of each measure, or datum, asses to its true value. True values for
many behaviors of interest to ABA are evident and universally accepted or can be
established conditionally by local context. True values for some behaviors are difficult
because the process for determining a true value must be different from the
measurement procedures used to obtain the data one wishes to compare to the true
value.
Why is the interobserver agreement very important to measurement? - ANS-To
determine the competence of new observers, detect observer drift, judge whether the
definition of the target behavior is clear and the system not too difficult to use, and
convince others of the relative believability of the data.
, List the requisites for obtaining valid IOA measures - ANS-It requires that two or more
observers use the same observation code and measurement system, observe and
measure the same participants and events, and observer and record the behavior
independent of influence by other observers.
If given some data, be able to calculate IOA using the preferred method of doing so,
based upon the type of measurement system used to collect those data. If given a
description of a target behavior and context, be able to select the best observational
recording system to use to measure that behavior. 114-119 - ANS-...
Summarize the considerations in selecting, obtaining, and reporting IOA information,
including how often IOA should be collected and acceptable levels of IOA - ANS-IOA
assessments should occur during each condition and phase of a study and be
distributed across days of the week, times of day, settings and observers. 80% is the
standard acceptable level of IOA.
accuracy - ANS-(of measurement) The extent to which observed values, the data
produced by measuring an event, match the true state, or true values, of the event as it
exists in nature.
believability - ANS-The extent to which the researcher convinces herself and others that
the data are trustworthy and deserve interpretation. Measures of interobserver
agreement are the most often used index of this in applied behavior analysis.
calibration - ANS-Any procedure used to evaluate the accuracy of a measurement
system and, when sources of error are found, to use that information to correct or
improve the measurement system.
continuous measurement - ANS-measurement conducted in a manner such that all
instances of the response class(es) of interest are detected during the observation
period. Frequency recording, duration recording
direct measurement - ANS-Directly measure the behavior of interest rather than
measuring indicators of the behavior. (Frequency data on nail-biting occurrences rather
than checking nails periodically).
discontinuous measurement - ANS-Measurement conducted in a manner such that
some instances of the response class(es) of interest may not be detected. Whole
interval, partial interval, momentary time sampling
What are the 3 indicators of "trustworthy" measurement? - ANS-Validity, accuracy, and
reliability.
Describe validity in terms of one of the 3 indicators of "trustworthy" measurement. What
does it measure? - ANS-Requires 3 equally important elements: (a) measuring directly
a socially significant target behavior, (b) measuring a dimension of the target behavior
relevant to the question or concern about the behavior and (c) ensuring that the data
are representative of the behavior's occurrence under conditions and during times that
are most relevant to the question or concern about the behavior.
Describe accuracy in terms of one of the 3 indicators of "trustworthy" measurement.
What does it measure? - ANS-Refers to the extent to which the observed value
matches the true state or true value of the event as it exists in nature. It must be
determined if there is a measurement bias.
Describe reliability in terms of one of the 3 indicators of "trustworthy" measurement.
What does it measure? - ANS-When measuring a procedure yields the same outcome
each time. It measures consistency of data collection.
How do we ensure our data are valid, accurate, and reliable? - ANS-Researchers and
practitioners who asses the accuracy of their data can (a) determine early in an analysis
whether the data re usable for making experimental or treatment decisions, (b) discover
and correct measurement errors, (c) detect consistent patterns of measurement error
that can lead to the overall improvement or calibration of the measurement system, and
(d) communicate to others the relative trustworthiness of the data.
What are some procedures that provide direct and indirect measurement? - ANS-Direct
observation is an example of direct measurement and questionnaires filled out by
parents is an example of indirect measurement.
Describe the difference between continuous recording and sampling recording. Give
examples of each, and advantages and disadvantages. How do we do each type of
recording? 106-107 - ANS-Continuous recording is considered the gold standard in
applied behavior analysis as it continuously records behaviors in a session. sampling
recording differs as data is recorded in samples or increments that have been
,predetermined. Sample recording may be utilized when an observer is collecting data
for multiple students at one time.
Name the three threats to measurement accuracy and reliability. - ANS-Human error is
the biggest threat to measurement accuracy and reliability. Factors that contribute to
human measurement error include poorly designed measurement systems, inadequate
observer training and expectations about what the data should look like.
poorly designed measurement systems - ANS-An example would be an unnecessarily
cumbersome and difficult to use measurement system which can create a needless loss
of accuracy and reliability.
inadequate observer training - ANS-observers who are not trained enough. Explicit and
systematic training of observers i essential for the collection of trustworthy data.
unintended influences on observers - ANS-Presuppositions an observer may hold about
the expected outcomes of the data and an observer's awareness that others are
measuring the same behavior.
How do we maximize the quality of observers who are taking our data? 108-109 -
ANS-Provide a systematic approach that covers methods such as sample vignettes,
narrative descriptions, video sequences, role playing and practice sessions int eh
environment
Describe how we assess the accuracy and reliability of data, including the differences
between obtained and true values of data. Describe how, if you can, look at data and
know whether those data are obtained or true or both? 110-11 - ANS-calculate the
correspondence of each measure, or datum, asses to its true value. True values for
many behaviors of interest to ABA are evident and universally accepted or can be
established conditionally by local context. True values for some behaviors are difficult
because the process for determining a true value must be different from the
measurement procedures used to obtain the data one wishes to compare to the true
value.
Why is the interobserver agreement very important to measurement? - ANS-To
determine the competence of new observers, detect observer drift, judge whether the
definition of the target behavior is clear and the system not too difficult to use, and
convince others of the relative believability of the data.
, List the requisites for obtaining valid IOA measures - ANS-It requires that two or more
observers use the same observation code and measurement system, observe and
measure the same participants and events, and observer and record the behavior
independent of influence by other observers.
If given some data, be able to calculate IOA using the preferred method of doing so,
based upon the type of measurement system used to collect those data. If given a
description of a target behavior and context, be able to select the best observational
recording system to use to measure that behavior. 114-119 - ANS-...
Summarize the considerations in selecting, obtaining, and reporting IOA information,
including how often IOA should be collected and acceptable levels of IOA - ANS-IOA
assessments should occur during each condition and phase of a study and be
distributed across days of the week, times of day, settings and observers. 80% is the
standard acceptable level of IOA.
accuracy - ANS-(of measurement) The extent to which observed values, the data
produced by measuring an event, match the true state, or true values, of the event as it
exists in nature.
believability - ANS-The extent to which the researcher convinces herself and others that
the data are trustworthy and deserve interpretation. Measures of interobserver
agreement are the most often used index of this in applied behavior analysis.
calibration - ANS-Any procedure used to evaluate the accuracy of a measurement
system and, when sources of error are found, to use that information to correct or
improve the measurement system.
continuous measurement - ANS-measurement conducted in a manner such that all
instances of the response class(es) of interest are detected during the observation
period. Frequency recording, duration recording
direct measurement - ANS-Directly measure the behavior of interest rather than
measuring indicators of the behavior. (Frequency data on nail-biting occurrences rather
than checking nails periodically).
discontinuous measurement - ANS-Measurement conducted in a manner such that
some instances of the response class(es) of interest may not be detected. Whole
interval, partial interval, momentary time sampling
