1.3 Research Design
We assume that the universe is orderly, and events have specific causes. We use the scientific method to study the universe
The General Procedures for the Scientific Method
Ask a question about the world and identify relevant terms needed to ask the question.
Operationalize the relevant terms. “Operationalize”: a concept defined by how we measure the terms of interest. Usually involves putting a number on an abstract concept, but doesn’t have to be a number either.
Pick a research method
Collect & analyze data
Research Methods
These differ in the kinds of information about behavior they yield, as well as in the types of behavior to which they are best suited for studying. We will look at five different methods. Note that they are not mutually exclusive.
- Observational It is a systematic method for observing behavior as it naturally occurs.
aka systemic observation or naturalistic observation. This method is characterized by:
- Unobtrusiveness - subjects are unaware that they are being observed
- Naturalness - subjects are “at home” and thus assumed to behave as natural as possible
- Systematic Recording - Behavior is measure and/or counted.
For example,
frequencies : how many
duration : how long
latentcies : how long until
etc. are recorded for operationally defined behaviors
Since researchers are observing and measuring behaviors, there might be differences between observers in agreement. The measure of how reliable operationalized measures are called, unsurprisingly, reliability.
- Surveys
Gathering a information through questionaires, interviews, etc. on a subset of population of interest. This method is characterized by:
- Sampling a subset of population and measuring these subset (since measuring the whole population is impossible and/or prohibitively costly)
Things to consider:
- Sampling a subset of population and measuring these subset (since measuring the whole population is impossible and/or prohibitively costly)
- Sampling adequacy
- Case Studies
An individual or small group of individuals of interest is studied in detail. These individuals or small group of individualsare called a cohort. This is characterized by two main ways of performing case studies:
- Retrospective - looks back at past events of the cohorts.
Note that we never measured anything in the past. We ask the individuals (and/or people around them, if relevant to the study) about the cohort’s past behaviors. The reliability of this sort of data also depends on the ability for the subject to recall memories. - Longitudinal or Proactive - follow events as they occur.
We first identify cohorts and continue to study the same cohort for some period of time. This sort of data are very accurate but very costly to do in large scale. Note that many individuals in the cohorts are expected to fall out of the study, so we would have to start with large population. If we are studying a condition that occurs at adulthood and are interested in measuring since birth but we don’t know if a baby would have this condition in the future, then we would have to appropriately have very large cohort to increase our likelihood of capturing individuals of interest by the end of the study.
For example schizophrenia that occurs about 1% of the population between age 15 and 35. We want to have 10 subjects with schizophrenia at the end of 35 year study. We should start with 1000 babies and hope for the best that no one drops out (unrealistic, for sure).
- Retrospective - looks back at past events of the cohorts.
- Experimental Method
This is what we most likely are talking about when we say an “experiment to test effect of X on Y”. If randomized control trial (RCT) is used, then this method can provide strong evidence for cause & effect.
- Independent Variable (IV) - Variable that we can select and manipulate.
IV must have at least two levels (categorical) or values (numerical) that IV can take. If IV has only one level or value, then that IV is not useful at all. - Dependent Variable (DV) - Variable that we measure and is interested in making inferences about. Sometimes callec criterion variable.
- Extraneous Variable (EV) - Variable other than IV that can influence the DV. If an EV effects the groups in an experiment (groups determined by the levels of IV, e.g.
treatmentIV with levelscontrolanddrug), then we do not know whether changes in DV are explained by IV or EV. Usually both IVs and EVs effect the variability in DV, and we call such Evs confounding variable. Table 1.1 shows possible effects of EV on DV. Pay close attention to the effect on groups created by IV (e.g.controlanddrug).
- Independent Variable (IV) - Variable that we can select and manipulate.
| Possible Effect on DV | Is it a problem? | Statistical term |
|---|---|---|
| EV has no effect on either groups | Not a problem | ? |
| EV effects all the groups in the same manner | Not a problem | fixed effect (?) |
| EV effects groups differentially based on which level the subject belongs to | Not a problem | interaction effect (?) |
PS Under certain designs, we could say IV effects DV with the word effect as a verb to mean IV “results in or brings about change or consequence” to note cause & effect as apposed to affect which is less stringent and to mean that IV “influences changes in” DV. Nitpicky for sure, but an important distinction. I am assuming that the experimental method would satisfy the condtions for the strong claim of effect.