1. Introduction

Statistics courses are mandatory in many social science study programs, including psychology.

Such courses are indispensable for a thorough scientific training in the empirical behavioural sciences. Unfortunately, many students experience anxiety in relation to learning statistics, i.e. statistical anxiety (Baloǧlu, 2003; Onwuegbuzie & Wilson, 2003), and this can pose a problem for both the students and instructors of statistics courses (Chew & Dillon, 2014; Ralston et al., 2016). Cruise et al. (1985) defined statistics anxiety as “the feeling of anxiety encountered when taking a statistics course or doing statistical analysis” (p. 92), while Zeidner (1991) described statistical anxiety more in depth as”… a particular form of performance anxiety characterised by extensive worry, intrusive thoughts, mental disorganisation, tension, and physiological arousal”. (p. 319)

Statistical anxiety in higher education has been linked to inadequate learning behaviour in students (e.g., Macher et al., 2011; Onwuegbuzie, 2004) and to both poorer academic outcomes (e.g., Kesici et al., 2011; Macher et al., 2011, 2013) and better exam results (e.g., Hunt et al., 2023). Similarly, positive attitudes towards statistics have been linked to high course grades (e.g., Fullerton & Umphrey, 2002), as non-related to statistical anxiety (e.g., Macher et al., 2011), and as a mediator of statistical anxiety (e.g., Sesé et al., 2015). Thus, while previous research does not point clearly to the consequences of statistical anxiety and/or attitudes towards statistics as being negative or positive in relation to learning and training in mandatory areas such as statistics in psychology education as problematic in itself. In any case, high arousal and a state of alert in the student body within a discipline do not enhance the learning environment or the well-being of students.

Several instruments have been suggested for measuring statistical anxiety and attitudes towards statistics, including the Statistical Anxiety Rating Scale (STARS; Cruise & Wilkins, 1980; Cruise et al., 1985), the Attitude Towards Statistics scale (ATS; Wise, 1985) and the Survey of Attitudes Towards Statistics (SATS; Schau et al., 1995). These instruments are all rather dated when considering the progress in opportunities to learn and work with statistics within higher education courses, e.g., on one's own laptop rather than in a laboratory, with open-source statistical software, or with support of an abundance of online help and tutorials. In this study, the newer Attitudes and Relationship to Statistics - Revised (Original Danish name “Holdninger og Forhold til Statistik”, HFS-R; Nielsen & Kreiner, 2018; 2021a; 2021b) instrument for measuring both statistical anxiety and attitudes towards statistics will be investigated. The FHS-R not only measures overall statistics anxiety, but also subtypes of this with particular relevance for the classroom context in higher education, such as test and classroom anxiety, interpretation anxiety, fear of asking for help, and attitudes towards worth of statistics for the academic discipline and future career.

The HFS-R has previously been studied in its original Danish version (FHS-R-DK; Nielsen & Kreiner, 2018; 2021a; 2021b), while studies with an English translation and a Ukrainian translation are ongoing in research groups in the UK and Germany, respectively. In this study, we investigate a German translation (HFS-R-G), as it compares to the Danish original version, administered to psychology students in the German-speaking part of Switzerland.

When an instrument, like the HFS-R, is translated into different languages, it is important to investigate for violations of measurement invariance before comparing the results across language forms (Merenda, 2005). In the validation of instruments by means of Rasch models (Rasch, 1960) items are routinely tested for differential item functioning (DIF; Holland & Wainer, 1993) to detect items that have different measurement properties for different subgroups of participants, as well as invariance of the total set of items in a scale across subgroups.

The current study is the first validity study investigating the measurement properties of the German language version of the HFS-R by comparative item analyses of a subsample of students from Zurich, in the German-speaking part of Switzerland, who completed the German language version, to a Danish subsample of students, who completed the original Danish language version. To fulfil this aim, we conducted extensive item analyses of the subscales in both languages focusing specifically on the issue of comparability across the two language versions (i.e. differential item functioning and overall invariance).

2. Methods

2.1 Instrument

The HFS-R questionnaire (In Danish: Holdninger og Forhold til Statistik – Revideret; Nielsen & Kreiner, 2021b) measures statistical anxiety as well as attitudes towards statistics. The HFS-R was developed and first validated in Danish (Nielsen & Kreiner, 2018) on the basis of a translation of the Statistical Anxiety Rating Scale (STARS; Cruise & Wilkins, 1980; Cruise et al., 1985). For details on how the HFS-R relates to the STARS, see Nielsen and Kreiner (2018). A second validity study has also been conducted in the Danish context (Nielsen & Kreiner, 2021a).

The HFS-R consists of 26 items comprising three subscales measuring anxiety and one subscale measuring attitudes: the Test and Class Anxiety scale (TCA) with seven items, the Interpretation Anxiety scale (IA) with eight items, the Fear of Asking for Help scale (FAH) with five items, and the Worth of Statistics scale (WS) with six items. For the three anxiety subscales, students rate the items according to the degree of anxiety they would experience in the described situations related to their statistics course using a four-point response scale: 1 = no anxiety, 2 = a little anxiety, 3 = some anxiety, and 4 = a lot of anxiety. For the attitude subscale, students are asked to state their agreement with each item statement using another four-point response scale: 1 = definitely disagree, 2 = disagree more than agree, 3 = agree more than disagree, and 4 = definitely agree.

For this study, the HFS-R was translated into German. The Danish version served as a basis for translating the HFS-R items, while an unvalidated English translation was used to clarify the meaning of items in the translation process. Two independent translations were done and systematically compared. Discrepancies were rigorously discussed to obtain a consensus version that all translators agreed upon .  All translators had a very good understanding of English. Three of the translators were native German speakers, one of which had a good understanding of Danish, and one had a good understanding of German while being a native Danish speaker. The items are shown in both Danish and German in the Appendix (Table A1).

2.2 Participants and data samples

All participants were university students enrolled in psychology programs and taking statistics courses within these programs (Table 1). In total, the data sample comprised 813 students; 445 Danish students and 368 Swiss students. The Danish subsample was collected during 2018-2019, while the Swiss subsample was collected in 2021.

The Danish subsample consisted of students from one Danish university taking statistics courses within the Bachelor of Psychology degree program. The courses were a first-semester and a second-semester statistics course, both consisting of a weekly lecture and small-group exercise classes. The Danish data were collected in the fifth lecture of the respective courses, thus one month into the first or the second semester of the degree program. Data were collected using a paper-and-pencil version of the Danish language version of the HFS-R questionnaire, including a number of demographic and other background questions. Students had been informed of the data collection in advance and that it was voluntary to participate. Time had been allowed for data collection within the specific lectures, specifically 15 minutes at the end of the first half of the lecture. At the time of the data collection, students were informed of the purpose of the overall project (i.e. to investigate statistical anxiety and attitudes towards statistics, including conducting various types of validity studies), that their data would be treated according to the prevailing data protection regulations, that they could ask to have data deleted until data had been anonymized, and that participation was voluntary. Students were provided the same information in writing as well as contact information for the principal investigator.  

In the Danish subsample, some students completed the survey only once, while some students completed the survey twice: 173 were enrolled in a first-semester course, and responded only once to the survey. Of these, 164 responded that this was their first statistics course within higher education, while nine responded that it was not. One hundred and fifty were enrolled in a second semester statistics course and responded only once to the survey. As the second-semester course required completion of the first-semester course, these students were not taking their first statistics course within higher education. One hundred and sixty-seven students responded in both their first-semester course and their second-semester course. Of these, six responded in the first semester that this was not their first statistics course in higher education. The remaining 161 cases were assigned randomly to represent either the group of students taking their first statistics course or the group not taking their first statistics course in the data sample, respectively. The resulting Danish data sample thus consisted of 445 students, with 55.5% taking their first statistics course in higher education and the remaining 44.5% taking a statistics course that was not their first. The Danish students were, on average, 22.9 years old (SD = 5.27) and mostly female (72.6%).

Similar to the Danish sample, the Swiss data were collected in the fifth lecture of statistics courses in the Bachelor of Psychology degree program. The courses were an introductory statistics course without computer exercises for 1^st semester students (as well as students repeating the class one year later), an introduction to statistical software for 3^rd semester students, and an intermediate-level statistics course for 5^th semester students. All three courses were mandatory. Accordingly, the Swiss subsample consisted of psychology students either in their 1^st, 3^rd or 5^th semester. In the lectures, the purpose of the study was briefly introduced in the last 15 minutes of the lecture, and students were then given access to the survey though a link to an online survey hosted by Limesurvey (LimeSurvey GmbH. (n.d.). The survey collected the socio-demographic information of the participants as well as some background questions (see Table 1) before showing the HSF-R items. Students were given 10 minutes time to reply to the survey using their private laptops or the cell-phones. The Swiss students only completed the survey once. In total, N = 368 students completed the HSF-R, and they were, on average, 22.6 years old (SD = 4.52) and mostly female (79.3%). The Swiss data sample consisted of 38% students, who were taking their first statistics course while the majority were taking a statistics course, which was not their first.

Only 7.6% of the Swiss students found that statistics were not relevant for their future employment. This percentage was notably lower than in the Danish sample, where 16.9% found that they would not be needing statistics for their future employment. The majority of the Swiss students deemed their mathematics level as adequate or more than adequate for learning statistics (88%), and the same was the case for the Danish students (74.6%). 

Table 1.

Characteristics of the Total Sample and the Danish and Swiss Subsamples

Notes. Percentages not summing to 100 within a variable and sample are due to missing responses. These were in general low (less than 10), except for the Danish subsample, where 50 students did not provide information on their perception of the adequacy of their mathematics level for learning statistics, and 63 students did not provide information on gender or age.The latter two groups are not identical.

2.3. Rasch and graphical log-linear Rasch models

Based on previous studies of the psychometric properties of the HFR-S (Nielsen & Kreiner, 2018, 2021a), we did not expect all subscales of the HFS-R to fit pure Rasch models (RM; Rasch, 1960), and thus we used both the RM and graphical log-linear Rasch models (GLLRM; Kreiner & Christensen, 2007), which are both latent variable models. The RM is a measurement model, which, in statistical terms, is a parsimonious model (Fischer & Molenaar, 1995) describing the causal effect of a latent trait variable on responses to items (Borsboom, 2005), and it has particularly desirable properties (see section 2.3.1). GLLRMs are extended and generalized RMs, which retain most of the properties of the RM (Kreiner & Christensen, 2007). Both the RM and GLLRMs belong to the larger family of Item Response Theory (IRT) models, as the RM can be considered a special case of these and GLLRMs extend the RM, though not by adding parameters in the usual manner to extend to 2PL or 3PL models (Christensen, Kreiner & Mesbah, 2013; Fischer & Molenaar, 1995; van der Linden & Hambleton, 1997). Contrary to other psychometric IRT and CFA models, neither the RM nor the GLLRM need assumptions on the distribution of the latent variable (Kreiner & Christensen, 2007; Rasch, 1966; Wright & Panchapakesan, 1969).

2.3.1. The Rasch model

The RM is one among several latent variable models which can provide valid measurement of unobservable (i.e. latent) traits. We find the RM to be the preferable point of departure in analyses, because it is a parsimonious model and because it is the only IRT model that may provide specific objective measurement (Fischer & Molenaar, 1995; Rasch, 1961). This renders the RM unique because it is the only model where the raw score over all items is a sufficient statistic for the person parameter estimates (Fischer, 1995). Sufficiency is an attractive property for several reasons: one is that it allows assessment of fit and estimation of item parameters that do not depend on a specific distribution of the latent variable, another is that the sum score contains all the information needed to assess the level of the latent construct measured and can be used, if preferred, instead of the person parameter estimates.

The requirements for fit of a set of item responses to the RM are (Kreiner, 2013; Mesbah & Kreiner, 2013):

1. Unidimensionality: The items of the scale must only assess one single underlying latent construct. In this case study: the TCA, IA, FAH and WS subscales assess four different constructs and separately they are unidimensional.
2. Monotonicity: The expected item scores will increase with increasing values of the latent variable.
3. Local independence of items (no local dependence; no LD): Responses to items should be conditionally independent from responses to any other item in the scale given the latent variable. In other words, local independence implies that responses to an item only depend on the level of the construct measured, and not on responses to other items.
4. Invariance and no differential item functioning (no DIF): Items and exogenous variables (i.e., background variables) should be conditionally independent given the latent variable. Thus, responses to an item must only depend on the level of the construct measured. Gender or other subgroupings of students may have a direct effect on the construct being measured, but there should be no effect of the background variables on the responses to items given the latent variable. For single items and a background variable, this is termed absence of DIF, while for the entire set of items and a background variable, this is termed invariance.
5. Homogeneity: the rank order of items by the expected item scores should be the same at all levels of the latent variable. In this case: the item requiring the least of the relevant construct to be endorsed (i.e., what we could term the easiest item or the item requiring the least anxiety to be endorsed) should be the same for students at a high level of the construct as for students at a lower level of the construct. In the same way, the ordering of all remaining items should be the same for all students no matter their level on the construct.

The first four requirements for fit to the RM meet Rosenbaum’s (1989) requirements for criterion-related construct validity, and thus we claim that RMs provide valid measurement. These requirements are common for all unidimensional IRT models. The fifth requirement of homogeneity is unique to the RM, and together with the first four requirements, this provides sufficiency of the sum score (i.e., the summed raw score is a sufficient statistic for the estimated person parameters, c.f. the above).

In this study, we used the Partial Credit model (PCM; Master, 1982) for ordinal categorical items, as this is a generalization of the RM for dichotomous items, and it provides the same measurement properties as the model for dichotomous items (Mesbah & Kreiner, 2013).

2.3.1. Graphical log-linear Rasch models

Measurement scales within the field of psychology are often challenged by DIF and/or local dependence (LD), as is evident also by the previous studies of the HFS-R (Nielsen & Kreiner, 2018, 2021a). To avoid eliminating items from already brief scales in order to obtain strict validity (c.f. the above requirements for the Rasch model), Kreiner and Christensen (2002, 2004, 2007) proposed a class of extended and generalized Rasch models referred to as graphical log-linear Rasch models (GLLRMs).

In GLLRMs, locally dependent items can be included as interaction terms, if the strength of the association between dependent items is constant across all levels of the latent variable. In the same way, interaction terms between items and background variables may be included in GLLRMS to account for DIF, if the direct effect of the background variable on the item is constant across all levels of the latent variable. DIF and LD satisfying these requirements are termed uniform DIF (Hanson, 1998), and likewise uniform LD. Kreiner (2007) and Kreiner and Christensen (2007) claimed that Rasch models with uniform DIF and uniform LD provide essentially valid and objective measurement (further information on essential validity is provided in supplemental file 2 in Nielsen & Kreiner, 2021a)^[1].

In chain graph models, nodes representing variables and edges and arrows are used to illustrate associations among the variables. Missing edges or arrows between nodes mean that the variables are conditionally independent given the remaining variables in the model. A directed edge (arrow) connecting two variables may refer to a causal relationship, and undirected edges illustrate that the variables are conditionally dependent without causality assumed (see Lauritzen, 1996, for a comprehensive introduction to graphical models). In GLLRMs, items follow the same rules as other variables in graphical models. Thus, items which are not connected by an edge in a GLLRM graph are conditionally independent given the latent variable (i.e., items are locally independent). Likewise, items and exogenous variables that are not connected by an arrow are conditionally independent (i.e., they do not function differentially). Items are connected to the latent variable to indicate that the latent variable drives responses to the items. In addition, if background variables, which are considered criterion variables, are directly associated with the latent variable in the expected manner, this indicates that measurement is criterion valid.

2.3.2. Item analysis by Rasch models and GLLRMs

A rigorous test of the fit of a set of items in a single scale to a RM or a GLLRM includes the following steps:

• Overall test of homogeneity of item parameters across low and high-scoring groups.
• Overall tests of invariance relative to background variables.
• Fit of the individual items to the RM.
• Tests of no DIF for all items relative to background variables.
• Tests of local independence for all item pairs.
• Tests of unidimensionality if subject matter considerations suggest that more than one latent variable could lie behind the item responses.

The steps do not need to be taken in the order presented above. If evidence of LD or DIF turns up, log-linear interactions are added to the model and the above steps repeated until no further evidence against fit to the model is disclosed.

When the final model is known, the following steps conclude the analysis:

• Targeting and reliability relative to the current study population are evaluated.
• Tests of unidimensionality across the three anxiety subscales are conducted to confirm that these measure different latent traits.

All statistics used test whether item response data comply with the expectations of the model, and so for all results significant p-values signify evidence against the model. In line with the recommendations by Cox and colleagues (1977), we evaluated p-values as a continuous measure of evidence against the null hypothesis, distinguishing between weak (p < 0.05), moderate (p < 0.01), and strong (p < 0.001) evidence against the model. In addition, we used the Benjamini-Hochberg (1995) procedure to control the false discovery rate (FDR) due to multiple testing to reduce the amount of false evidence. 

The fit of individual items to the RM (or subsequently a GLLRM) was tested by comparing the observed item-restscore correlations with the expected item-restscore correlations under the model (Christensen & Kreiner, 2013; Kreiner, 2011). Overall tests of homogeneity (i.e., are the item parameters the same for persons scoring high and low, respectively) and invariance (i.e., are the item parameters for the total set of items in a scale the same for persons belonging to subgroups) were conducted using Andersen’s conditional likelihood ratio test (CLR; Andersen, 1973). The local independence of items and absence of DIF was tested using Kelderman’s (1984) likelihood-ratio test, and if evidence against these assumptions were discovered, the magnitude of the local dependence of items and/or DIF was informed by partial Goodman-Kruskal gamma coefficients conditional on the restscores (Kreiner & Christensen, 2004). DIF-analyses and analyses of invariance were done in relation to six background variables: Sample (Danish, Swiss), Statistics course at university level (1^st statistics course, not 1^st statistics course), Perceived adequacy of mathematics level to learn statistics (more than adequate/adequate, less than adequate), Expectation to need statistics in future employment (yes, maybe, no), Gender (female, male), and Age group (20 years and younger, 21 years, 22 years and older).

If DIF was present, the sum scores were equated for DIF so that the direct effect of the background variable causing DIF would be taken into account, thus making the scale scores comparable across the relevant subgroups. To calculate such comparable scores, we first estimated the person parameters for the DIF subgroups based on the different sets of item parameters resulting from the DIF (also referred to as splitting for DIF; Hagquist et al., 2009). We then selected one subgroup as the reference and calculated the expected score for the remaining subgroups as if they had belonged to the reference group (i.e., that there was no DIF and the item parameters were identical for the subgroups) (Kreiner & Nielsen, 2023). 

Reliability was calculated using Hamon and Mesbah’s (2002) Monte Carlo method, which takes into account any local dependence between items. Targeting^[2] was assessed numerically with two indices (Kreiner & Christensen, 2013): the test information target index (the mean test information divided by the maximum test information) and the root mean squared error target index (the minimum standard error of measurement divided by the mean standard error of measurement). Both indices should have a value close to one. We also estimated the target of the observed score and the standard error of measurement of the observed score. Lastly, to provide a graphical illustration of targeting and test information, we plotted person-item maps showing the distribution of the person parameter estimates against the item category thresholds, with the inclusion of the test information curve.

Unidimensionality across the three anxiety subscales (TCA, IA & FAH) was tested by pairwise comparisons of the observed and expected gamma (γ) correlations under the model (Horton et al, 2013). Scales measuring different constructs will be less strongly correlated than what is expected under the common unidimensional model.

2.4 Software

All item analyses were conducted with the DIGRAM software package (Kreiner, 2003; Kreiner & Nielsen, 2013, 2023), while the item maps were generated using R.

3. Results

We report the main results for the four subscales in this section, while some additional results are provided in the supplemental file. All items are referred to by their original item number in Nielsen and Kreiner (2018), in order to make comparisons to other studies straightforward.

After exclusion of item TCA8 (Going through an exam assignment in statistics after the grade has been given)^[3], item IA8 (Seeing a fellow student concentrating on their output from statistical analyses)^[4] and item WS8 (Statistics provides the most objective and firm knowledge)^[5], the reduced TCA (6 items), IA (7 items) and WS (5-items) subscales each fitted a graphical log-linear Rasch model (GLLRM) with DIF as well as locally dependent items (Figure 1). The full 5-item FAH subscale also fitted a GLLRM with DIF and locally dependent items (Figure 1). When taking into account the DIF and local dependence within each subscale, there was no evidence against overall homogeneity (i.e. no difference in item parameters across low and high-scorers) or against overall invariance or any additional DIF across subgroups defined by Sample, Course number, Perceived adequacy of mathematics level  to learn statistics, Expectancy to need statistics in future employment, Gender or Age groups (Table 2), nor was there any evidence against the fit of individual items (Table A4 in the Appendix).

The GLLRMs for all subscales included some locally dependent items (see section 3.1. for details) and DIF relative to sample for up to four items. The FAH and IA subscale GLLRMs additionally included DIF relative to Course number, while the WS subscale GLLRM included DIF relative to Perceived adequacy of mathematics level to learn statistics (Figure 1, see section 3.2 for further details). Results of the CLR tests confirming that the DIF and local dependence interaction terms should be included in the respective models are provided in Table A4, and item category thresholds, difficulties, and targets are provided in Table A5 in the Appendix.

The results of equating sum scores for DIF are shown in Tables A6 to A9 in the Appendix, while the assessment of the effect of DIF is shown in section 3.2 (Tables 3 to 6).

3.1 Local dependence

The analyses revealed evidence against conditional independence of some items in all four subscales of the HFS-R (Figure 1 & Table A4). In the TCA subscale, five items were found to be locally dependent with moderate (items TCA2 and TCA7), strong (TCA2 and TCA4), and very strong (TCA1 and TCA3) partial g correlations (Figure 1, top left). The first locally dependent pair of items was TCA2 (Doing the homework for a statistics course) and TCA7 (Attending lectures in statistics), the second pair was TCA2 (c.f. above) and TCA4 (participating in statistics exercises), and the third pair was TCA1 (Studying for an examination in a statistics course) and TCA3 (Doing the final examination in a statistics course).

In the FAH subscale, three items were pairwise locally dependent with moderate (FAH2 and FAH3) to strong partial g correlations (FAH1 and FAH2, FAH1 and FAH3, respectively) (Figure 1, top right). The locally dependent item pairs were: FAH2 (Asking for help with statistical analyses in class) and FAH3 (Asking one of your tutors for help in understanding a printout), FAH1 (Going to ask my statistics teacher for individual help with material I am having difficulty understanding) and FAH2, and FAH1 and FAH3.

In the IA subscale, two pairs of items were locally dependent to a moderate (IA2 and IA5) or strong (IA5 and IA6) degree (Figure 1, bottom left). The locally dependent items were IA2 (Making a decision based on statistical analyses) and IA5 (Trying to understand the output from statistical analyses), and IA5 and IA6 (Interpreting the meaning of a probability value once I have found it).

Finally, in the WS subscale, a single pair of items showed local dependence to a moderate degree (Figure 1, bottom right): WS5 (Statistics is useful) and WS7 (Statistics is an indispensable part of my academic program).

3.2. Effect of differential item functioning

Evidence of differential item functioning was found for all four subscales of the HFS-R, however, in relation to different background variables and of differing strength (Figure 1 and Table A4 in the Appendix) and differing impact on subsequent group comparisons (Table 3 to 6). Sample DIF (Danish versus Swiss) was found in all four subscales, DIF relative to course number (first statistics course versus not first statistics course) was also found in the FAH and IA subscales, while DIF relative to Perceived adequacy of mathematics level to learn statistics (more than adequate/adequate versus less than adequate) was found in the WS subscale.

Notes. γ correlations are partial Goodman and Kruskal’s rank correlation for ordinal data. Math = Perceived adequacy of mathematics level to learn statistics. Stat = Expectation to work with statistics in the future. Course = first statistics course or not.

Figure 1.

The Final Models for the Four HFS-R Subscales. Test and Class Anxiety (Top Left), Fear of Asking for Help (Top Right), Interpretation Anxiety (Bottom Left), Worth of Statistics (Bottom Right).

Table 2.

Global Tests of Homogeneity and Invariance for the Final HFS-R Graphical Loglinear Subscale Models in Figure 1

Notes. TCA = Test and Class Anxiety. FAH = Fear of Asking for Help. IA = Interpretation Anxiety. WS = Worth of Statistics. CLR = Conditional Likelihood Ratio test. Subsamples = the Danish and Swiss subsamples. Course number = is the statistics course the 1^st or not the 1^st at university level. Math adequacy = Perceived adequacy of mathematics level to learn statistics. Stat future employment = Expectancy to need statistics in future employment.  

^a  The TCA model assumes that items 1 and 3, 2 and 4, and 2 and 7, respectively, are locally dependent, and that items 3, 4, 5 and 7 functions differentially for Subsamples.

^b The IA model assumes that items 5 and 6 are locally dependent, that item 5 functions differentially relative to course number, and that items 1, 3 and 6 functions differentially for Subsamples.

^c  The FAH model assumes that the items 1 and 2, and 2 and 3, respectively, are locally dependent, that item 1 functions differentially relative to Course number, and that items 1, 3, 4, and 5 functions differentially for Subsamples.

^d  The WS model assumes that item 5 and 7 are locally dependent, and that item 7 functions differentially for Subsamples and groups defined by Perceived adequacy of mathematical level for learning statistics.

^e The test of homogeneity is a test of the hypothesis that item parameters are the same for persons with low or high scores.

⁺ The Benjamini-Hochberg adjusted critical level for false discovery rate at the 5% level was p = .0071.

3.2.1 The Test and Class Anxiety subscale

Four items of the TCA subscale functioned differentially relative to Sample (Figure 1 and Table A4 in the Appendix); TCA3 (Doing the final examination in a statistics course) and TCA7 (Attending lectures in statistics) with the Danish students more likely to report high levels of anxiety in relation to these activities, no matter their level of TCA. Swiss students were more likely to report high levels of anxiety withTCA4 (Participating in statistics exercises) and TCA5 (Finding that another student in class got a different answer than you did to a statistical problem), regardless of their level of TCA.  No DIF was discovered for any of the other included background variables.

The overall effect of the sample DIF on the TCA sum score is shown in Table 3, which also shows the test bias that would result if the sum scores were not equated for DIF. The largest bias of approximately one third of a point was observed for the group of students perceiving their mathematics level to be less than adequate to learn statistics. The conclusions from the subgroup comparisons were the same whether the observed sum scores or the DIF-equated sum scores were used in the comparisons, and thus the effect of the DIF was not large enough to lead to statistical errors in this sense, though the difference in the mean scores of the Danish and Swiss sample was slightly reduced.

Table 3.

Comparison of Observed and DIF-equated Mean TCA Sum Scores for Subgroups of Students

Notes. ^a. all subgroups differ

3.2.2 The Fear of Asking for Help subscale

Four items in the FAH subscale functioned differentially relative to Sample (Figure 1 and Table A4 in the Appendix); FAH1 (Going to ask my statistics teacher for individual help with material I am having difficulty understanding), FAH3 (Asking one of your tutors for help in understanding a printout), FAH4 (Asking for an explanation of something I do not understand in statistics lectures), and FAH5 (Asking a fellow student for help in understanding output). For FAH1 and FAH 4, the DIF was such that the Danish students were more likely to report high levels of fear of asking for help, regardless of their level of FAH, while the Swiss students  were more likely to endorse  items FAH3 and FAH5. In addition, item FAH1 also functioned differentially relative to course number (i.e. whether students were taking their first statistics course or not). No DIF was discovered for the remaining background variables.

Table 4 shows that substantial test bias would resul,t if the sum scores were not equated for the discovered DIF for almost all subgroups (more than half a point on the scale), and this is more severe when comparing the Swiss students to the Danish students (-1.20 points on the scale). Thus, while the overall conclusions of group comparisons were the same whether the observed sum scores or the DIF-equated sum scores were used, the substantial bias resulting from a failure to DIF-equate the FAH scores would provide results that would be substantially off-mark in subgroup comparisons. Again, as for the TCA subscale, differences in mean scores are reduced in size as a result of the DIF-equating, and in the case of perception of mathematics level, the evidence of a difference in FAH scores for the two groups was weakened when DIF-equating was performed (i.e., p value changed om 0.010 to 0.028).

Table 4.

Comparison of Observed and DIF-equated mean FAH Sum Scores for Subgroups of Students

Notes. ^a. all subgroups are equal

3.2.3 The Interpretation Anxiety subscale

As for the FAH subscale, there were items that functioned differentially relative to Sample and course number in the IA subscale (Figure 1 and Table A4 in the Appendix). No DIF was discovered for the remaining background variables. Three items function differentially relative to Sample: IA1 (Interpreting the meaning of a table in a journal article), IA3 (Interpreting the meaning of a probability value once I have found it), and IA6 (Interpreting the meaning of a probability value once I have found it), with the Danish students being more likely to report higher levels of anxiety in relation to these activities, no matter their level of IA. In addition, IA5 (Trying to understand the output from statistical analyses) functioned differentially relative to course number (i.e. whether student were taking their first statistics course or not): so that students taking their first statistics course were more likely to report high anxiety, no matter their level of IA.

Table 5 shows that, again, substantial test bias would result, if the sum scores were not equated for the discovered DIF for almost all subgroups (more than half a point on the scale in most cases), and the most severe for the Swiss students compared to the Danish students (-0.95 points on the scale). Further, in this case, a Type I error would ensue if DIF equating was not performed, as we would falsely claim that there was a difference in the Danish and Swiss students’ levels of Interpretation Anxiety, when this was not the case if DIF was accounted for. For the remaining variables defining subgroups, the overall conclusion of comparisons would not differ across the observed and the DIF-equated scores. However, the evidence of a difference in Interpretation Anxiety between the younger and the older group of students is strong when scores have been equated for DIF, while only weak if the DIF had not been taken into account.

Table 5.

Comparison of Observed and DIF-equated Mean IA Sum Scores for Subgroups of Students

Notes. ^a. all subgroups differ

3.2.4 The Worth of Statistics subscale

In the WS subscale, evidence of DIF was found for item WS7 (Statistics is an indispensable part of my academic program) both in relation to Sample and whether students perceived their mathematics level as adequate or not for learning statistics (Figure 1 and Table A4 in the Appendix). The Swiss students and students who perceived their mathematics level as less than adequate for learning statistics being more likely to agree with the items, no matter their level on the WS scale. No DIF was discovered for the remaining background variables.

Table 6 shows that some test bias would result if the sum scores were not equated for the discovered DIF for almost all subgroups (up to half point on the scale), with the most severe bias being for the group of students perceiving their level of mathematics to be less than adequate for learning statistics (0.54), and the Swiss students (0.44). The overall conclusions of group comparisons were the same whether the observed or the DIF-equated sum scores were used. Differences in mean scores were reduced in size as a result of the DIF-equating, and in the case of previous courses in statistics, the evidence of a difference in WS scores between students in their first or a later statistics course disappeared.

Table 6.

Comparison of Observed and DIF-equated Mean WS Sum Scores for Subgroups of Students

Notes. ^a. all subgroups differ

3.3 Targeting and reliability

Table 7 summarizes the results of the analysis of targeting and reliability. Targeting was more than adequate for most subgroups in the four HFS-R subscales with a few exceptions: The targeting of the TCA subscale was, however, poor for the Swiss sample of psychology students, with only 40% of the maximum test information obtained. Likewise, the targeting of the FAH subscale was less than adequate (just below 60% information obtained) for the Danish sample no matter if they were taking their first or a subsequent statistics course. The targeting of the IA and WS subscales were quite similar for the Danish and Swiss samples in terms of the information achieved. When comparing the target values with the mean scores (theta or the sum scores), as an expression of targeting, there were differences between the sample subgroups, particularly for the IA and the FAH subscale, where these values are closer together for the Danish students compared to the Swiss students (Table 7). Such differences and similarities in targeting arise naturally as a result of the combination of differences in levels on the measures and differences in item difficulty due to i.e. DIF, and are thus not surprising. 

The item maps (Figures A1 to A4 in the Appendix) illustrate both the level of information along the scales, the point of maximum information (the target), and the degree of alignment of the person estimates and the item thresholds. Figure A1 shows that students in the Danish sample are located evenly across the range with item thresholdsalong the TCA scale and that information is relatively high along the scale. However, many students in the Swiss sample are located where there is no item information, thus the information is quite low along the majority of the scale. The poorer targeting of the FAH for the Danish students compared to the Swiss students is illustrated by the poorer alignment between items and persons along the scale, as the items have the most information at the higher end and the Danish students are located more toward the lower end of the scale in comparison to the Swiss students (Figure A2). 

The reliability of the four subscales of the HFS-R was satisfactory for research and statistical purposes for all subgroups except two: the TCA for the sample of Danish psychology students (r = 0.62) and the WS subscale for the group of Danish students, who found their level of mathematics inadequate for learning statistics (r = 0.46). The item maps for the TCA and the WS subscales (Figures A1 and A4 in the Appendix), illustrate the reason for the low reliabilities for the Danish students, as there is substantially less variation in the person estimates than for the Swiss students and particularly so on the WS subscale for the Danish students who perceived their mathematics level to learn statistics to be inadequate.

Table 7.

Targeting and Reliability of the Test and Class Anxiety, Fear of Asking for Help, Interpretation Anxiety and Worth of Statistics Subscales

Notes. TI = test information, RMSE = The root mean squared error of the estimated theta score. SEM = The standard error of measurement of the observed score. r = reliability, Stat course = statistics course, Math adequate = Mathematics level perceived as adequate to learn statistics, Math not adequate = Mathematics level perceived as inadequate to learn statistics.

^a Targeting and reliability are provided for subgroups where items functioned differentially.

^b Weighted average reliability across DIF-subgroups: TCA r = 0.68, FAH r = 0.77. IA r = 0.78. WS r = 0.72.