Investigating the Validity and Reliability of Survey Attitude towards Statistics Instrument among Rural Secondary School Students

Positive attitude towards statistics is a desirable aspect in developing statistical thinking and understanding statistical concept. The attitude towards statistics instrument in previous study on rural school students lacked evidence in the validity analysis. Therefore, there is a need to provide a valid instrument that can assess rural secondary school students’ attitude toward statistics. For this purpose, this study was carried out to investigate the validity and reliability analysis of the Survey Attitude towards Statistics (SATS-36) instrument from a sample of rural secondary school students. The SATS-36 is a 7-point Likert type scale questionnaire, used to measure students’ attitude towards statistics. The SATS-36 instrument was distributed to 217 Tenth grade science stream students from a rural district in Sabah, Malaysia, which comprised of 66 (30.4%) boys and 151 (69.6%) girls. A second order Confirmatory Factor Analysis (CFA) was carried out to analyze the validity of the SATS-36 instrument as well as to confirm the 6-factor model of SATS-36. The reliability of SATS-36 instrument was assessed through the Cronbach’s Alpha value. Overall, the analysis showed that the SATS-36 instrument shows favorable indices for both its validity and reliability. Thus, it can be concluded that the SATS-36 was a valid and reliable instrument for measuring students’ attitude towards statistics among rural science stream secondary school students.


Introduction
Developing a positive attitude towards statistics among the students is one of the main goals in statistical teaching. After all, learning attitudes towards statistics have regularly been reported to have a crucial role in students' mastery of statistics. It was hypothesized that students with positive attitude towards statistics would perform better in the statistical performances (Awaludina, Ab Razak, Harris & Selamat. 2015;Chiesi & Primi, 2015;Naccache, 2012). Generally, attitude is defined as a settled way of thinking or feeling about something -it can also be referred to the belief or feelings towards an individual, idea, object, matter or event. Meanwhile, attitude is defined by Fishbein and Ajzen (2010) as a tendency to react on a psychological object with some degree of favorableness or unfavorableness.
In statistics education, students' attitude towards statistics referred to the degree of students' positive and negative feelings towards the statistical subjects, which is related to the value and importance, self-efficacy and difficulty, and overall impression toward the subject (Evans, 2007). Meanwhile, according to Vanhoof, Kuppens, Sotos, Verschaffel and Onghena (2011), attitudes toward statistics is a multidimensional aspect referring to separate, but related tendencies associated to the favorable or unfavorable reactions concerning to statistics. Martins, Nascimento and Estrada (2011) described attitudes towards statistics as intense feelings which are relatively stable and results from positive or negative experiences encountered whilst learning statistical subject over a period. Emmioglu and Capa-Aydin (2012) on the other hand defined attitude towards statistics as tendencies of the students to respond negatively or positively to statistics.
According to Rahnaward, Ab Hamid and Zakaria (2015), it is important for the students to have a positive attitude towards learning since it enables the students to master the learning subjects. Besides that, many statistics instructors and students also believe that positive attitudes toward statistics are crucial in the statistical learning process (Mahmud & Osman, 2010). The assessment on students' attitude towards statistics is also important since it could motivate the students to improve their statistical skills, understand statistical concepts, and value the statistical knowledge in their everyday lives (Mohd Judi, Sahari, Mohamed & Tengku Wook, 2011a). Meanwhile Zhang, Shang, Wang, Zhao, Li, Xu and Su (2012) argued that students' attitudes towards statistics could influence their development by applying such statistical knowledge in their careers as well improving their skills in statistical thinking. Students who possess negative attitudes towards statistics would face a problem in learning the subject efficiently In Malaysia, there are considerable numbers of researches which have been conducted to investigate students' attitude towards statistics; however, most of the research were performed on the students from the higher level of education while research on secondary school students have been modest. Compared to students in higher institutions of learning who might encounter at least one statistics course during their study, less attention has been shed on secondary school students' attitude towards statistics. This is because statistics is viewed as part of the mathematical syllabus in the Malaysian secondary school mathematics curriculum. Thus, it is apparent that the assessment on the students' attitude towards mathematics was prioritized in comparison to the assessment on the students' attitude towards statistics. While in fact, there were evidences from the past addressed that the secondary school students in Malaysia did not show a good performance related to statistical learning, including the students who studied in the rural area (IEA, 2015;Saidi & Siew, 2019). Thus, the assessment on the attitude towards statistics should be carried out more comprehensively among the Malaysian secondary school students, since this factor might attribute to this matter. Previously, Saidi and Siew (2019) found that the rural secondary students' attitude towards the effort in learning statistics had significant influence on their understanding of statistical concepts, particularly in the measures of central tendency.
In 2010, Mahmud and Osman initiated a research on students' attitudes towards the learning of statistics among Malaysian secondary school students. They previously adapted the SATS instrument called Students' Attitude towards Learning Statistics to investigate the change in 237 eleventh graders' attitudes towards statistics and discovered that the students were slightly more positive at the end of the learning of elementary statistics compared to the beginning of the lesson. Similarly, Saidi and Siew (2019) also discovered a positive attitude towards statistics among the 148 Tenth grade science stream students who studied in a rural district of Sabah, Malaysia as overall, as well as in all of the components in SATS-36, except for the Value component. The students were found to react most positively in the Effort component, followed by Cognitive Competence, Interest, Difficulty, Affect, and Value components.This suggested several matters: students were willing to use more effort to learn statistics and believed in their cognitive ability to understand statistics; students also implied that statistics is not a difficult subject and that they had a high interest in it.
The aforementioned studies which have been conducted to assess students' attitude towards statistics at the Malaysian secondary school level adapted the SATS-36 instrument. Mahmud and Osman (2010)'s instrument, Students' Attitude towards Learning Statistics, is a 5-point Likert scale ranging from 1 (Strongly Disagree), 3 (Neutral) to 5 (Strongly Agree) and consists of 16 items. In addition to that, the reliability and validity of this instrument was tested through the Rasch Measurement Model (Che Musa, Mahmud, & Md Ghani, 2011). Meanwhile, the study of Saidi and Siew (2019) retained most of the original items in the SATS-36 instrument to assess rural secondary school students' attitudes towards statistics, but this study lacked evidence in the instrument's validity analysis. As both studies did not employ a Confirmatory Factor Analysis (CFA) as a tool of validation, thus, this encouraged the researchers to conduct a validity assessment on the SATS-36 instrument by using a CFA method based on the sample of rural secondary school students in Sabah.
According to Ishak (2012), 70% of Sabah's area is a hilly area, and about 60% of its population is scattered in the rural areas, while 40% are in the urban areas. There are also small islands along the coast, with 70% of them having a school. Meanwhile, there are about 80% of the schools are located in the rural areas, and the education achievement between the rural and urban schools in Sabah has a gap due to the shortage access to good teaching and learning resources.
Even on a global platform, studies conducted on attitudes towards statistics were primarily concentrated on the students in higher institutions who have taken statistics courses in their university, but there is sparsely any information on the attitudes towards statistics among younger students (e.g. students from preschool, primary and secondary school), as well as graduates who have completed their education (Schau, Millar & Petocz, 2012). Thus, more research is needed to discover students' attitudes towards statistics in these particular groups, and this study aims to fill the research gaps by assessing secondary school students' attitudes towards statistics, particularly in the rural area.
Besides that, even though the SATS-36 instrument is a renowned instrument to measure attitude towards statistics, it is noted that the validation of this instrument was executed using a post-secondary school sample. Meanwhile, in the assessment studies of validity and reliability analysis of SATS-36, most of the studies also used the post-secondary school sample (Coetzee, & Van der Merwe, 2010;Mohamad Judi, Mohamed, Sahari & Tengku Wok, 2011b;Tempelaar, van der Loeff, & Gijselaers, 2007;Vanhoof et al., 2011), and little evidence demonstrated the validity and reliability assessment by using the secondary school sample. Hommik and Liuk (2017) previously adapted SATS-36 instrument in order to assess 1357 secondary school students' attitude towards statistics within the Estonian educational context. Based on the Confirmatory Factor Analysis (CFA), it was discovered that the theoretical framework of SATS-36 instrument only fit four components instead of the original six components. Hommik and Liuk (2017) study provided a useful reference for other researchers in the process to develop a valid and reliable instrument based on the sample from secondary school students, but further studies need to be carried out to explore if the secondary school students from other sample exhibit similar results. Thus, the validation of the SATS-36 instrument by using the sample from the secondary school students in this study could be established and this validation enhanced this research.

Research Location and Sample
This research was carried out at Kota Belud, a rural district in Sabah, Malaysia. Sabah is a state located in the eastern part of Malaysia, with an area of 73 620 km 2 . In Sabah, there are 156 secondary schools categorized as rural schools, and these schools are from various District Education Offices. Meanwhile, in the Kota Belud district particularly, there are 9 secondary schools located in this area, and all of these 9 secondary schools offer the science stream class to the students, and all of these secondary schools are identified as rural schools. In Malaysian educational system, students at the upper secondary level who are academically inclined can choose either Arts stream or Science stream. Students who studied in the science stream classes are more exposed to the statistical contents and mathematics related subject.
In this study, the Kota Belud district was particularly chosen as a research location because there were evidences from the previous study which discovered that the students performed quite poorly in the study related to the statistical learning in this area, and these statistical performances could be influenced by their attitude towards statistics (Saidi & Siew, 2019). Thus, as the attitude was found to have influence on the students' statistical performances, this study attempted to provide a valid and reliable instrument that can measure the rural secondary school students' attitude towards statistics.
The sample of the study consisted of 217 Tenth grade science-stream students which comprised of 66 (30.4%) boys and 151 (69.6%) girls whose ages are between 15 to 16 years old. The students were randomly selected from the 9 secondary schools in Kota Belud by using simple random sampling technique. Thus, all of the students in the population have an equal chance to be selected. The random selection of 217 students with a population of 435 was done by using a random number generator. Schau (2003) which is used to measure students' attitudes towards statistics. It comprises of six components, which are: 1) Cognitive Competence: students' attitudes about their intellectual knowledge and skills related to statistics; 2) Value: students' attitudes towards the worth, relevance, and usefulness of statistics in career and daily life; 3) Difficulty: students' attitudes on the difficulty of statistics as a subject; 4) Affect: students' feelings concerning statistics; 5) Effort: the amount of effort the students are willing to invest in to study statistics; and 6) Interest: the level of students' interest in statistics.

SATS-36 instrument is a well-known instrument developed by
The SATS-36 instrument is a 7-point Likert type scale questionnaire. It consists of 36 items, where the responses range from 1 (strongly disagree) through 4 (neither disagree nor agree) to 7 (strongly agree). For analysis purposes, the negative wordings items were transformed to positive wordings in order to avoid the confusion held by the students when interpreting the reverses worded items (van Sonderan, Sanderman, & Coyne, 2013). Besides that, since the SATS-36 instrument was originally developed for post-secondary students, it was imperative to confirm whether this instrument was suitable to be used on the sample of the study. Due to the differences in the learning context between students in secondary school and higher level of education, some of the words in the original items may not suitable for the secondary school students. Thus, some items in the instrument were slightly revised so that it could be understood by secondary school students.   1 shows the revised items in the SATS-36 instrument. The word of 'course' in the original item C2, A5 and E2 is not suitable to be used for the sample of the study since in Malaysian secondary school's context, statistics is not offered as a single course, but is integrated as a part of mathematical topics or subjects in the Mathematics syllabus. So, the words 'topics' or 'subject' is more suitable to be used for those items. Other than that, the words of 'formula', 'valuable', and 'homework' were used for item C5, V1, and E1 respectively since the secondary school students in the study were more familiar with these words as compared to the 'equation', 'worthy', and 'assignment'. The content validity of this instrument was examined by an expert in mathematics education who had experience teaching upper secondary mathematics in school for almost 20 years.  Additionally, the SATS-36 instrument that was used in this study contained a Malay translation since the teaching of Mathematics in Malaysian secondary schools is in the Malay language. . The instrument's translation into the Malay language was done by the back-translation method. The instruments were translated from English into Malay language first by the researcher with the help of an English educator who was an expert in both Malay and English language. This resulted in a Malay version of the instruments. Next, the instruments were translated back by another person into English. Then, to ensure that the meaning of the items remained the same, the two English versions were compared.

Confirmatory Factor Analysis (CFA)
Confirmatory Factor Analysis (CFA) is a method of testing how well the measured variables represent a smaller number of constructs (Hair, Black, Babin & Anderson, 2014). In other words, the CFA method provides information of how well the prior theoretical measurement of the factors matched the reality (actual data). In this study, a second order CFA was employed by using the AMOS version 22 software to confirm whether the data could fit the 6-factor model of the SATS-36 instrument as hypothesized by Schau (2003). According to Chen, Sousa, and West (2005), the second order CFA is commonly used for instruments which measures several related constructs, where each of the construct is measured by multiple items, and this provides a more interpretable and parsimonious model for models with higher order factor as compared to the first order models with correlated factors. Meanwhile, Erol and Senturk (2018) recommended of using the second order CFA since certain sub-factors in the first order models might be unrelated to the general structure that needs to be assessed.
According to Awang (2012), there are three types of validity can be established through CFA method, which are construct validity, convergent validity, and discriminant validity. Construct validity is used to assess how well a measurement measures what it is supposed to measure. Construct validity as defined by Hair et al. (2014) is "the extent to which a set of measured variables actually represents the theoretical latent construct those variables are designed to measure (p. 601)". The construct validity through CFA can be assessed based on the goodness of fit, which provides useful information to assess whether the data could fit well with the measurement model that was hypothesized by previous researchers. The overall model fit can be assessed based on several fit indices. According to Hair et al. (2014), the fit indices should include at least one absolute fit index and one incremental fit index, in addition to the chi-square results.
As suggested by Hair et al. (2014), there are three types of fits which are included in the analysis. They are: 1) Absolute fit: Root Mean Square of Error Approximation (RMSEA) and 2) Incremental fit: Comparative Fit Index (CFI), Incremental Fit Index (IFI), Tucker Lewis Index (TLI); and 3) Chi-square: the relative chi-square (CMIN/DF). According to Hair et al. (2014), reporting these fit indices values usually provides sufficient information to evaluate a model. In order to achieve the model fit, Schumacker and Lomax (2004) suggested that the RSMEA with a value less than 0.05 is an indication of a good fit, while values between 0.05 and 0.08 can still be accepted. Meanwhile, CFI, TLI, and IFI should be more than 0.90 to be considered as an acceptable fit. For the CMIN/DF, the value must be within 1 and 5. To illustrate more, Table 3 presents the fit indices to assess the goodness of fit.  (2004) Meanwhile, convergent validity assesses whether the measures of a specific construct converge or share a high proportion of common variance. Hair et al. (2014) asserted that a high factor loading is an initial indication of a high convergent validity since they converge on a common point that is, the latent construct. The factor loading which is in the standardized form should be more than 0.50, while a factor loading higher than 0.70 is preferable. However, Awang (2012) asserted that the items which are newly developed should have factor loadings not less than 0.50, while items which have been established previously must be more than 0.60. Since the SATS-36 instrument was an established instrument, the factor loadings for these two measurements model should be more than 0.6. The items with low factor loading should be deleted one at a time and run the CFA again for the new measurement model.
Another way to assess the convergent validity through CFA is by using Average Variance Extracted (AVE). AVE explains the percentage of variation which can be obtained from the finalized items of the respective constructs and can be calculated by summing the squares of the factor loading and divided by the number of items in the particular construct (Maat, Zakaria & Rosli, 2017). In order to achieve convergent validity, the AVE value should exceed 0.50 (Awang, 2014;Hair et al., 2014). In addition to AVE, the Composite Reliability (CR) which assesses the internal consistency of the latent constructs should be more than 0.60 and is considered as acceptable and signals a good rate (Awang, 2014).
Discriminant validity on the other hand determines whether the construct is sufficiently distinct from other constructs. According to Awang (2014), discriminant validity is established when there are no redundant items in the measurement model that is, no items seem to be alike but measures different constructs (Hamdi & Bahruddin, 2015). The redundant items can be identified through the Modification Indices (MI), where a high value of MI (above 15) indicates that the two pairs of items are redundant, and one of the two items with the lower factor loading can be deleted from the measurement model. Another requirement for discriminant validity is the correlation between latent constructs should not exceed 0.85. The correlation which exceeds 0.85 indicates that the two latent constructs are redundant or having serious multicollinearity issues (Awang, 2014).

Cronbach's Alpha Value
The reliability of the SATS-36 instrument will be determined by the Cronbach's Alpha value. George and Mallery (2003) provided the rule of thumb for assessing the Cronbach's Alpha value for a dichotomous or Likert scale instrument. This is presented in Table 4. The Cronbach's Alpha value will be analyzed by using SPSS version 20.0.  (2003) The Cronbach's Alpha has a value between 0 and 1. The closer the Cronbach's Alpha value to 1, the greater the internal consistency of the item within the scale. According to George and Mallery (2003), Cronbach's Alpha value above 0.90 indicates excellent internal consistency, above 0.80 is good, above 0.70 is acceptable, above 0.60 is questionable, above 0.50 is poor, and below 0.50 is unacceptable.   0.80 Figure 1 shows the original measurement model of SATS-36. Meanwhile, Table 5 shows the standardized factor loading for all items in the SATS-36 instrument. Since the SATS-36 instrument is an established instrument, the factor loading should be more than 0.60 to achieve a better model fit. Thus, based on Table 5, items V1, V2, D5, D6, D7, and A2 are candidates for removal due to the low factor loading which is less than 0.60 (in bold).The item with the lowest factor loading was deleted first one at a time, and then the CFA was conducted again from the start.  The finalized items in the SATS-36 instrument after the deletion of items are shown in Table 6 and Figure 2. Besides that, Table 6 also shows the Average Variance Extracted (AVE) value and Composite Reliability (CR) value for the components in the final model of SATS-36. It was discovered that all the components in the final model had an AVE value which exceeded 0.50; this indicated convergent validity evidence. Moreover, all the CR value exceeded 0.80 which is higher than the acceptable value ( 0.60) as suggested by Awang (2014), which indicated that the internal consistency of the components in the final model is good and acceptable. Meanwhile, Table 7 refers to the goodness of fit indices for the original and final model of SATS-36. It was discovered that the final model of SATS-36 has an acceptable fit for all the fit indices assessing the overall model fit. To compare the models, the Akaike Information Criterion (AIC) was used, where the model with the lowest AIC was preferred and considered as the better one (Kline, 2005). Based on the AIC value in Table 7, it can be concluded that the final model of SATS-36 which contains of 30 items is a better model than the original model of SATS-36.  Table 8 shows the correlation among the components in the final model of SATS-36, with the value in the diagonal position refers to the square root of AVE value (in bold). Based on Table 8, it is apparent that the inter-correlation among the components is significant. Besides that, no correlation among the components exceeded 0.85, and the square root AVE value is higher than the values in its column and row. This indicates that there is an evidence of discriminant validity for all the components in the final model of SATS-36. Other than that, the assessment from the Modification Indices (MI) also confirmed the discriminant validity of SATS-36 since it was discovered that there were no pair of items in the final model had high MI value above 15.  Hommik and Liuk (2017) who adapted the SATS-36 for Estonian secondary school students discovered that the 4-factor model of SATS-36 had a better goodness of fit, where the Cognitive Competence, Difficulty and Affect components were also integrated into one component called Competence. In this study, there were several components found to be highly correlated with each other. So, it might be possible to integrate the highly correlated components into one component and assess for another factor solutions, but this analysis was not further explored in this study. As for the reliability assessment, the findings were in line and supported the findings from the other studies. According to Schau (2003), the reliability assessment from the other studies usually discovered that the overall of SATS and its components had reasonably high Cronbach's Alpha values which indicated good internal consistencies.

Conclusion
In summary, this study provides the validity and reliability evidence of the SATS-36 instrument based on the sample from the rural secondary school students in Kota Belud, Sabah, Malaysia. Overall, the instrument has excellent reliability with a Cronbach's Alpha value of 0.958, proving that the SATS-36 is a reliable instrument for measuring students' attitudes towards statistics among the sample of the study. Meanwhile, through the analysis using the CFA method, this study confirmed the 6-factor model of the SATS-36 model which is based on the sample of the study. However, due to low factor loading, six items were excluded from the instrument, which were items V1 (Statistics is valuable), V2 (Statistics should be a required part of my professional training), D5 (Statistics doesn't involve massive computation), D6 (Statistics is highly technical), D7 (Most people don't have to learn a new way of thinking to do statistics), and A2 (I will feel secure when statistics problem). These items might not be relevant for the sample of the study, but further investigation needs to be carried out to confirm this matter. Overall, the analysis from CFA also confirms that the SATS-36 instrument has a construct validity, convergent validity and discriminant validity evidence based on the sample of the study.

Suggestions
Through this study, the validity and reliability of SATS-36 was established for rural secondary school students. For future research, the researchers suggested to conduct the validity and reliability of SATS-36 for rural secondary school students in other countries. According to Hommik and Liuk (2017), it is important for secondary school students to feel positively about statistics before entering colleges since the statistics course at college can sometimes be an unexpected topic for the students. Thus, the study of validation of the SATS-36 instrument on Malaysian secondary school students would greatly contribute to the field of research on rural secondary school students' attitudes towards statistics in other countries. To date, research has scarcely investigated secondary school students' attitude towards statistics in Malaysia (Mahmud & Osman, 2011;Saidi & Siew, 2019). This provides an opportunity to carry out similar researches that are targeted especially at the rural secondary school samples.