Introduction
Reliability analysis provides information on how reliable an analysis is. It describes the extent to which a procedure measures identically when performed repeatedly. If I run my experiment a second time, will comparable results come out? If the answer is “no” it means: we have a problem. Perfect reliability, on the other hand, means that a measurement (experiment, questionnaire) would always produce the identical results as long as the conditions do not change. A low reliability causes different results under the same conditions.
What is Cronbach’s alpha in reliability analysis?
Cronbach’s Alpha is a common reliability coefficient used to assess the internal consistency or reliability of measurements. It is often used to measure the reliability of scales or questionnaires that consist of multiple items. Cronbach’s alpha is calculated from the correlation coefficient between items in a scale and indicates a value between 0 and 1, with a higher value indicating higher reliability.
However, there are some limitations to the use of Cronbach’s alpha, particularly when assessing scales with few items or when items do not correlate well with each other. Nevertheless, Cronbach’s alpha is a useful tool to assess the internal consistency of measurements and can be useful in many disciplines. In this introduction, we use Cronbach’s alpha.
Where can I find the sample data for this tutorial?
The sample data is available here: Sample data
The calculation of reliability with SPSS with Cronbach’s alpha
- Wir klicken auf Analysieren > Skala > Reliabilitätsanalyse ….
In the dialog box we see two columns. On the left are the variables available in the data set.
In this window we select the variables for the reliability analysis
We click on a variable to the right as part of the reliability analysis. As long as all variables are on the right side
To calculate the Cronbach’s alpha, we need variables. We put them into the right window.
Further down, Alpha must be selected as the model so that the Cronbach’s Alpha is calculated. 3.
Then click on the Statistics… button. button is clicked.
In the field Descriptive statistics for we click on the options Item and Scale if item deleted. In the Between items field we select Correlations.
Then we confirm the entries by clicking on the Next button.
We then click on Ok and start the calculation with SPSS.
3. Interpret output
SPSS now gives us several tables that we will analyze. Let’s start with the valid cases in the Case Processing Summary table. In our example, no case was excluded and all cases are valid (valid, N=1158.).
Let’s move on to the most important table: the output of Cronbach’s alpha. In the first column “Cronbach’s Alpha” the value .851 is written, in the second column we can see the Cronbach’s Alpha calculated from correlations, in column three the number of items is written (this is the number of variables studied).
What do we do with the result? In general, the higher the Cornbach’s Alpha value, the better. The following table shows the quality of the Cronbach’s Alpha value:
| Chronbach´s Alpha | Evaluation |
|---|---|
| > 0.9 | excellent |
| > 0.8 | Good or high |
| > 0.7 | acceptable |
| > 0.6 | questionable |
| > 0.5 | bad |
| ≤ 0.5 | unacceptable |
This table gives an overview of the interpretation of the Cronbach’s Alpha. In our sample data set is .859 and according to the rated as “good”. The larger the value, the better. Anything below .5 is too little and anything below .8 can at least be discussed critically.
Negative Cronbach’s Alpha
Sometimes the analysis gives a negative Cronbach’s Alpha value, which is not good. If this happens, we should take another look at the inter-item correlation matrix. Maybe we forgot to invert items. That means that negated questions in the data set have not been changed or there are items in the data set that say something opposite.
Next step. We look at the summary table named “Item Statistics” with the columns Mean, Std. DeviationStandardabweichung Die Standardabweichung ist ein Maß für die Streuung der Werte einer Variablen um ihren Mittelwert und gibt an, wie sehr die Werte von ihrem Durchschnitt abweichen. Sie wird häufig verwendet, um die Varianz innerhalb einer Population oder Stichprobe zu beschreiben und kann verwendet werden, um die Normverteilung einer Variablen zu beschreiben. Eine kleine Standardabweichung bedeutet, dass die Werte der Variablen dicht um ihren Mittelwert clustern, während eine große Standardabweichung darauf hinweist, dass die Werte der Variablen weiter verteilt sind. (SD) and Number of Cases (N).
By looking at this table, we can see the average response behavior for each question. We keep in mind our scale used. It can range from 1 to 10, as in our case. Satisfaction with assortment (m=4.54, sd=1.36) has the lowest average value, while satisfaction with user guidance (m=6.1, sd=0.81) is the highest.
Inter-Item-Korrelationsmatrix
This table shows us the correlations between the individual items. We make sure that no items have too high a correlation. If two or more items have too high a correlation, this is an indicator of multicolinearity. This means that both items are too similar and basically carry almost the same information. No value should have a correlation r=.7 or higher! If this is the case, the corresponding item should be excluded if necessary. In our example, the correlation between “Satisfaction Overall Shopping Experience” and “Satisfaction Online Store Design” stands out with the correlation of r=.63. The correlation is high, but still within the bounds.
Item-Skala-Statistiken (Item-Total-Statistics)
Note: If an exclusion of a variable changes the value only insignificantly, the item should rather remain and not be removed.
We stay in the table and look at the column squared multiple correlation and gives us the explanatory variance (R²), which we know from the regression analysis. SPSS does a lot of the work for us and calculates a separate multiple regression for each item, using the other variables as predictors, inheriting the weaknesses of regression analysis: the more items, the higher R² turns out to be.
Ergebnisse beschreiben
Bei der Berechnung der internen Konsistenz wurde ein positiver Cronbach´s Alpha Wert von .851 errechnet. Das ist ein Indikator für eine gute interne Konsistenz.
Conclusion on Reliability Analysis in SPSS
Overall, reliability is an important concept in statistics that describes the stability or reliability of measured values. Reliability analysis can be used to assess the reliability of measurement instruments or methods and is especially important when making comparisons between groups or over time.
There are several reliability coefficients that are appropriate depending on the type of measurement and the data available, and it is important to consider the strengths and weaknesses of the various coefficients when assessing reliability. By looking carefully and critically at reliability, one can ensure that the measured values are reliable and valid, and thus suitable for analysis and interpretation of the data.
5 Facts about Reliability Analysis in SPSS
- Reliability refers to the stability or reliability of measured values.
- A measurement instrument or method is reliable if it would produce the same results if used more than once.
- Reliability is important when making comparisons between groups or over time.
- There are different reliability coefficients that are appropriate depending on the type of measurement and the data available.
- The strengths and weaknesses of the various reliability coefficients should be carefully considered when assessing the reliability of measurements.
Reliability test procedure
| Art der Reliabilität | Testverfahren |
|---|---|
| Interrate reliability | Cohens Kappa |
| Internal consistency | Cohens Alpha |
| Comparison of test methods | Cohens Kappa |
| Retest-Reliabilität | ◊ Paralleltest-Reliabilität ◊ Split-Half-Reliabilität oder Testhalbierungsmethode ◊ Interne Konsistenz |
Häufig gestellte Fragen und Antworten: Reliabilitätsanalyse in SPSS
What is reliability analysis?
Reliability analysis is a statistical technique used to evaluate the reliability or stability of measurements or tests. A measurement or test is reliable if it produces the same results when performed multiple times.
Test-retest reliability: this type of reliability is used to evaluate the stability of measurements or tests over time. For example, one might examine the test-retest reliability of a personality test by administering the test to the same individuals at two different times and then comparing how similar the results are.
Inter-rater Reliability: this type of reliability is used to evaluate the agreement between two or more raters taking the same measurements or tests. For example, one might examine the inter-rater reliability of school performance assessments by assigning two or more teachers to assess the same students and then comparing how similar their assessments are.
Parallel Form Reliability: This type of reliability is used to assess the stability of measurements or tests over time. It is used when there are two or more similar versions of a test or measurement called parallel forms. Parallel forms reliability is particularly useful when using tests or measurements that are administered over a long period of time, for example, in education or psychological diagnosis.
What does Cronbach Alpha say?
Cronbach Alpha is a measure of the internal consistency or reliability of measurements or tests. It is used to evaluate how well the questions or items included in a test work together to measure a particular construct or concept.
Cronbach Alpha is calculated on a scale of 0 to 1, with a higher value representing higher internal consistency or reliability. A Cronbach Alpha value of 0.7 or higher is generally considered acceptable, while a value of 0.8 or higher is considered very good.
Cronbach Alpha is particularly useful when using tests or measurements that consist of multiple questions or items, for example in education or psychological diagnosis.
When to use Cronbach’s alpha?
Cronbach Alpha can be used to assess whether a test or measurement is reliable enough to serve as a basis for important decisions, for example in personnel psychology selection or in the diagnosis of mental disorders.
It is important to note, however, that Cronbach’s alpha is only a measure of internal consistency and does not indicate whether the test or measurement is valid, that is, whether it actually measures what it claims to measure. Validity is a separate property that must be examined in other ways.
What to do if Cronbach’s alpha is bad?
If the Cronbach’s Alpha values of a test or measurement are below the acceptable level of 0.7, some steps could be taken to improve internal consistency or reliability:
Review questions or items: it might be helpful to review the questions or items in the test or measurement to ensure that they all contribute to a consistent concept or construct. It might also be useful to review the questions or items to make sure they are understandable and free of errors.
Removing questions or items: it might be useful to remove questions or items from the test or measurement that do not fit well with the other questions or items or that reduce internal consistency.
Improving instructions and administration: it might be useful to revise the instructions for the test or measurement to ensure that they are clear and understandable, and to improve the administration of the test or measurement to ensure that it is conducted consistently.
It is important to note, however, that improving the internal consistency or reliability of the test or measurement does not automatically mean that it is valid, i.e., that it actually measures what it claims to measure. Validity is a separate property that must be examined in a different way.
What are further links to Check Reliability in SPSS
An article from Gesis.org: here
Wikipedia article about reliability: here
A definition from Spektrum: here
Methods portal of the University of Leipzig: here
What influences reliability?
The reliability of measurements or tests can be influenced by various factors. Here are some examples:
Quality of questions or items: de quality of questions or items included in a test or measurement can affect reliability. Questions or items that are unclear, flawed, or do not contribute to a consistent concept or construct can reduce reliability.
Instructions and administration: the quality of the instructions and administration of the test or measurement can affect reliability. If instructions are unclear or if the test or measurement is not administered consistently, this can reduce reliability.
Test conditions: Test conditions can affect reliability. For example, distractions or confounding factors during testing could reduce reliability.
Population: the composition of the population for which the test or measurement was developed can affect reliability. If the test or measurement has not been validated for the population for which it is being used, this may reduce reliability.
It is important to note that reliability is only a property of measurements or tests and does not indicate whether they are valid, i.e., whether they actually measure what they claim to measure. Validity is a separate property that must be studied in a different way.
