Experimental design

Japanese: 実験計画法 - じっけんけいかくほう（英語表記）experimental design

In experimental research, the method for determining a research design that can adequately answer the research interest, as well as the analysis method after data has been acquired, is called experimental design. Experimental design includes not only making accurate guesses about factors that the researcher can manipulate or control, but also data collection methods to eliminate the influence of uncontrollable factors. In experimental design, independent variables and covariates are called factors or factors. Factors are usually categorical variables, and each category is called a level. For example, if the experimenter can manipulate the type of stimulus and the teaching conditions, which are factors that affect the rate of correct answers, the experimental design will be a two-factor one, and if there are three types of stimuli, the level of the stimulus factor will be three.

[Fisher's three principles] Before explaining specific experimental design methods, we will explain the three principles that experimental design should have, as stated by Fisher, RA, the founder of statistical experimental design. The first is randomization, which randomly allocates the factors of interest to eliminate the influence of other factors (called covariates or confounding factors) and increase the internal validity of the study. The second is replication, which obtains multiple data for each level of the factor in question in order to know the magnitude of error and test whether the effect of the factor is statistically significant. The third is local control, which states that factors that cannot be manipulated but have a large effect on the dependent variable should be actively included in the analysis. Specifically, randomization and replication are performed within each level of the factor that should be locally controlled. Factors used for local control are sometimes called block factors. Block factors are usually factors that cannot be controlled by random allocation, such as personal factors when conducting repeated experiments on the same subjects. Analyses are often conducted assuming that there are no interactions with other factors.

[One-way layout model and two-way layout model] When there is only one factor of interest, a one-way layout model using that factor is used. In this case, the data is expressed as the overall mean, the main effect of the level to which the subjects belong, and the sum of the error. However, to make the model identifiable, constraints such as the sum of the main effects being zero are usually imposed. A test that assumes that the main effects of each level are all equal is called a main effect test, and if this is rejected, it is considered that the factor had an effect on the dependent variable. In addition, after it is found that the main effect test results are significant, a comparison between levels is usually made, in which case a multiple comparison method that takes into account the multiplicity of tests is used.

On the other hand, when there are two factors of interest, or when there is one factor other than the variable of interest that may affect the dependent variable, a two-way layout model is used. The data are expressed as the overall mean, the main effect of the level of factor 1, the main effect of the level of factor 2, the interaction term, and the sum of the errors. The difference from the one-way layout model is that there is an interaction term that represents the effect of a combination of specific levels of two factors on the dependent variable. If this exists, it is not very meaningful to discuss the effect of each factor separately. Therefore, first, a test is performed to see if there is an interaction, and if it is significant, a test of the simple main effect, which represents the effect of one factor at a specific level of the other factor, is performed. Since the test of the simple main effect is performed repeatedly, the significance level is often adjusted using a multiple comparison method. On the other hand, if the test of the interaction term is not significant, the main effects of the two factors can be analyzed in the same way as the one-way layout model.

It is also possible to consider a multifactorial model, which handles three or more factors simultaneously, in a similar way; however, this is not often used because it requires consideration of higher-order interaction terms and the statistical power is low unless the sample size is large.

[Various experimental design methods] The randomized block design is a design when a block factor exists, and was originally used in agriculture to consider plots that affect dependent variables such as yield as block factors and perform local management within the blocks. When the block factor is the experimental participant, it is specifically called a repeated measurement design. In a repeated measurement design, paired data is obtained for the same subjects for the number of levels of the experimental factor. In this case, a two-way model without replication is applied, and interactions between the experimental factor of interest and the block factor are not present or are considered to be simply errors.

In addition, when subjects are randomly assigned to each level of factor A and then repeated measurements are performed on factor B, a case in which paired and unpaired data are mixed is called a split plot design.

When there are multiple factors, the number of measurements becomes enormous if all combinations between levels are considered and further replicated. Therefore, a research design is required to investigate the main effects and interaction effects of the factors of interest with as few experiments as possible, and the Latin square design, which can significantly reduce the number of experiments when no interactions exist, and fractional factorial experiments using orthogonal arrays, which further refine the idea and enable analysis of more factors and levels with fewer experiments, are often used.

In order to conduct an analysis that satisfies the research interest, it is called factorial arrangement to decide what factors to consider, at what levels, how many times to repeat, what to use as a blocking factor, what fractional implementation method to use, etc. It is often used in engineering, but it is also useful in psychology research when there are many factors to consider at the same time or when conducting exploratory research.

[Repeated measures analysis of variance] When the same subjects are repeatedly measured for multiple experimental factors, a multivariate analysis of variance model can be implemented for the dependent variables for the number of levels of the experimental factors. For example, if there are four learning conditions, a four-variate multivariate analysis of variance model can be used to test whether there is a difference between the means of the four variables, assuming that there is a correlation between the four variables. However, if it is possible to make a sphericity assumption (or sphericity assumption) for correlation (strictly speaking, covariance), it is possible to test the main effect of the experimental factors by ignoring the correlation and using a randomized block design (two-way analysis of variance model without repeated measures) in which subject factors are added as block factors to the experimental factors. Furthermore, it is known that two-way analysis of variance is better than multivariate analysis of variance in terms of statistical power, so a test for the sphericity assumption is first performed, and if the assumption holds, a two-way analysis of variance is often performed.

[Fixed effects and random effects] When the main effects and interaction effects at each level or cell are unknown parameters and are to be estimated or tested, they are called fixed effects, and the corresponding factors are called fixed factors. The main effects and interaction effects of experimental factors are fixed effects. A model consisting only of fixed effects is called a fixed effects model.

On the other hand, there are cases where we are not interested in the differences in the levels of factors that need to be considered in experimental design. For example, when considering learning effects, a two-way model is applied with the test scores of multiple participants under various learning conditions as the dependent variables. Here, of the two factors, "differences in participants" and "differences in learning conditions," if the goal is not to know the aptitude treatment effect of each subject, but to identify conditions that generally improve learning effects, we are not interested in the main effects of each subject. In this case, the subjects who actually took the survey were a very small portion selected at random from the population, and we are not interested in the realized values of the main effects of each subject, so the issue is what the population distribution of the test scores of the subjects is like (for example, how large the population variance of the subjects is). In this way, when the main effect or interaction effect of a certain factor is considered to be a random variable, they are called random effects, the corresponding factors are called random factors, and a model consisting of only random effects is called a random effects model. In contrast, a model that has both random and fixed effects, such as the example where the "subject learning effect" is a random effect and the "effect of the learning condition" is a fixed effect, is called a mixed effects model. Generally, the block factor is often used as a random effect, and the power of the test for fixed factors is likely to be high.

The analysis of variance model used in the analysis of experimental design can be considered as a regression analysis model in which dummy variables exist as explanatory variables, but this can be understood in a unified manner as a generalized linear model, including ordinary regression analysis models. →Causal analysis →Regression analysis →Experimental method →Multiple comparisons →Statistical inference [Takahiro Hoshino]

Latest Sources Psychology Encyclopedia Latest Psychology Encyclopedia About Information

Japanese:

実験研究において研究関心に適切に答え得る研究デザインを決定するための方法，またデータを取得した後の解析法を実験計画法とよぶ。研究者が操作・制御できる要因について精度の高い推測を行なうことだけでなく，制御できない要因の影響を除去するためのデータ収集法も実験計画法に含まれる。実験計画法では独立変数や共変量を要因あるいは因子factorとよぶ。要因は通常カテゴリカル変数であり，各カテゴリーを水準levelとよぶ。たとえば正答率に与える要因のうち，刺激の種類と教示条件を実験者が操作できる場合，2要因の実験計画法になり，刺激の種類が三つならば刺激要因の水準は3となる。

【フィッシャーの3原則】　具体的な実験計画法について説明する前に，統計的実験計画法の創始者フィッシャーFisher,R.A.が挙げた実験計画法がもつべき3原則を説明する。まず一つ目は，関心がある要因について，無作為に配置を行なうことで他の要因（共変量や交絡因子とよばれる）の影響を除外し，研究の内的妥当性を高める無作為化randomizationである。二つ目は，誤差の大きさを知り，要因の効果が統計的に有意に大きいかについての検定を行なうために，当該要因の各水準について複数のデータを得る反復replicationである。三つ目は局所管理local controlで，操作できない要因のうち従属変数に対して影響が大きいものについては，積極的に解析で取り上げるべきであるというものである。具体的には，局所管理を行なうべき要因の各水準内で，無作為化と反復を行なう。局所管理を行なうために利用する因子のことを，とくにブロック因子block factorとよぶことがある。ブロック因子は通常無作為割当などの制御ができない因子であり，たとえば同一対象者に繰り返し実験を行なう場合の個人要因がこれに当たる。他の因子と交互作用が存在しないと考えて解析を行なうことが多い。

【一元配置モデルと二元配置モデル】　関心のある要因が一つだけの場合，それを要因とした一元配置モデルone-way layout modelが利用される。この場合，データは全体平均と，対象者が属する水準の主効果main effect，および誤差の和として表現される。ただしモデルの識別性のために，通常は主効果の和がゼロなどの制約をおく。各水準の主効果がすべて等しいことを帰無仮説とする検定を主効果の検定とよび，これが棄却された場合に，要因の従属変数への効果があったと考える。また，主効果の検定結果が有意であることがわかった後は，通常は水準間の比較を行なうが，その場合には検定の多重性を考慮した多重比較の手法を利用する。

　一方，関心のある要因が二つある場合，あるいは関心のある変数以外に従属変数に影響を与えうる要因が一つある場合には，二元配置モデルtwo-way layout modelを利用する。データは全体平均，要因1の水準の主効果，要因2の水準の主効果，交互作用項interaction term，誤差の和として表現される。ここで一元配置モデルとの相違は，2要因の特定の水準の組み合わせが従属変数に与える効果を表わす互作用項が存在することであり，これが存在する場合にはそれぞれの要因単独の効果を別々に議論することにはあまり意味がない。そこでまずは交互作用が存在するかどうかの検定を行ない，それが有意であれば一方の要因の特定の水準での他方の要因の効果を表わす単純主効果simple main effectの検定を行なう。単純主効果の検定については繰り返し検定を行なうことから，多重比較の方法で有意水準の調整を行なうことが多い。一方，交互作用項の検定で有意でない場合は，二つの要因の主効果について一元配置モデルと同様の解析を行なえばよい。

　三つ以上の要因を同時に扱う多元配置モデルも同様に考えることが可能であるが，高次の交互作用項について考察する必要があること，サンプルサイズを多くしないと検定力が低くなってしまうことから，あまり利用されない。

【さまざまな実験計画法】　乱塊法デザインrandomized block designはブロック因子が存在する場合のデザインであり，もともとは農学などで収量など従属変数に影響を与える区画をブロック因子として考え，ブロック内で局所管理を行なうために利用された。ブロック因子が実験参加者である場合には，とくに反復測定デザインrepeated measurement designとよばれる。反復測定デザインでは，同じ対象者について対応のあるデータが実験要因の水準数分得られることになる。この場合，繰り返しのない二元配置モデルが適用され，関心のある実験要因とブロック因子との交互作用は存在しない，あるいは単なる誤差として考えることになる。

　また，ランダムに要因Ａの各水準に被験者が割り当てられた後，要因Ｂについては反復測定がされる場合のように，対応のあるデータと対応のないデータが混在するような場合を分割区画デザインsplit plot designとよぶ。

　要因が複数存在する場合には，すべての水準間の組み合わせを考え，さらに反復を行なうとなると測定数が膨大になる。そこで，なるべく少ない回数で関心のある要因の主効果や交互作用効果を調べるための研究デザインが求められるが，交互作用が存在しない場合に回数を大幅に減少させることができるラテン方格法Latin square designや，そのアイデアをさらに洗練させ，少ない実験回数でより多くの要因や水準についての解析を可能にする直交表orthogonal arrayを用いた部分実施要因実験fractional factorial experimentがよく利用される。

　研究関心を満たす解析を行なうために，どのような要因について何水準で，どの程度繰り返しを行なうのか，ブロック因子として何を利用するのか，どのような部分実施法を利用するのかなどを決めることを要因配置計画factorial arrangementとよぶ。工学などではよく利用されるが，同時に考慮する要因が多い場合や探索的な研究を行なう場合には心理学研究でも有用である。

【反復測定分散分析】　同一対象者が繰り返し複数の実験要因について測定を受ける場合には，その実験要因の水準数分の従属変数に対する多変量分散分析モデルを実施すればよい。たとえば学習条件が四つあれば，4変量の多変量分散分析モデルを利用して，その4変量に相関があることを想定したうえで，4変量の平均値間に差があるかどうかを検定することができる。ただし相関（厳密には共分散）に球形仮定（または球面性仮定）sphericity conditionをおくことが可能な場合には，相関を無視して実験要因にブロック因子として対象者要因を加えた乱塊法（繰り返しのない二元配置分散分析モデル）を用いることで，実験要因の主効果の検定を行なうことができる。さらに検定力という観点からも，二元配置分散分析を行なった方が多変量分散分析よりもよいことが知られているため，まず球形仮定に関する検定を行ない，仮定が保持されれば二元配置分散分析を行なうことが多い。

【固定効果と変量効果】　各水準や各セルでの主効果・交互作用効果が未知の母数であり，推定，あるいは検定を行なうべき対象である場合，それらを固定効果fixed effectとよび，対応する要因を固定因子fixed factorとよぶ。実験要因での主効果や交互作用効果は固定効果である。固定効果だけで構成されたモデルを固定効果モデルとよぶ。

　一方，実験計画法で考慮する必要がある要因の各水準の差そのものに関心があるわけではないという場合もある。たとえば学習効果について考える場合，複数の参加者のさまざまな学習条件でのテスト得点を従属変数とした二元配置モデルが適用される。ここで二つの要因である「参加者の違い」と「学習条件の違い」のうち，各被験者の適性処遇効果を知ることが目的ではなく，一般に学習効果を向上させる条件の特定が目的ならば，個々の被験者ごとの主効果には関心はない。この場合，実際に調査を受けた被験者は母集団からランダムに選び出されたごく一部であり，個々の被験者の主効果の実現値自体には関心はないため，被験者のテスト得点の母集団分布がどのようなものであるか（たとえば被験者の母分散がどれくらい大きいか）といったことが問題となる。このように，ある要因の主効果や交互作用効果を確率変数と考える場合，それらを変量効果random effect，対応する要因を変量因子random factorとよび，変量効果だけで構成されたモデルを変量効果モデルとよぶ。これに対し，例に挙げたような「被験者の学習効果」を変量効果，「学習条件の効果」を固定効果とするなど，変量効果と固定効果を両方もつモデルを混合効果モデルとよぶ。一般にブロック因子を変量効果とする場合が多く，固定要因についての検定力も高くなる可能性が高い。

　実験計画法で解析に利用される分散分析モデルは，説明変数にダミー変数が存在する回帰分析モデルと考えることができるが，これは通常の回帰分析モデルを含めて一般化線形モデルgeneral linear modelとして統一的に理解することが可能である。　→因果分析　→回帰分析　→実験法　→多重比較　→統計的推論
〔星野崇宏〕

出典　最新心理学事典最新心理学事典について　情報

<<: Experimental Workshop

>>: Experimental cinema