Hostname: page-component-8448b6f56d-qsmjn Total loading time: 0 Render date: 2024-04-25T05:36:05.478Z Has data issue: false hasContentIssue false
  • English
  • Français

Measurement Invariance Study of the Training Satisfaction Questionnaire (TSQ)

Published online by Cambridge University Press:  12 June 2013

Susana Sanduvete-Chaves ,
F. Pablo Holgado-Tello ,
Salvador Chacón-Moscoso  and
M. Isabel Barbero-García
Susana Sanduvete-Chaves*
Affiliation:
Universidad de Sevilla (Spain)
F. Pablo Holgado-Tello
Affiliation:
Universidad Nacional de Educación a Distancia (Spain)
Salvador Chacón-Moscoso
Affiliation:
Universidad de Sevilla (Spain)
M. Isabel Barbero-García
Affiliation:
Universidad Nacional de Educación a Distancia (Spain)
*
*Correspondence concerning this article should be addressed to Susana Sanduvete-Chaves. Facultad de Psicología. Dep. Psicología Experimental, Área de Metodología de las Ciencias del Comportamiento. Universidad de Sevilla. C/ Camilo José Cela, s/n. Campus Ramón y Cajal. 41018 Sevilla (Spain). Phone: +34-954557715. Fax: +34-954551784. E-mail: sussancha@us.es
Get access Rights & Permissions [Opens in a new window]

Abstract

This article presents an empirical measurement invariance study in the substantive area of satisfaction evaluation in training programs. Specifically, it (I) provides an empirical solution to the lack of explicit measurement models of satisfaction scales, offering a way of analyzing and operationalizing the substantive theoretical dimensions; (II) outlines and discusses the analytical consequences of considering the effects of categorizing supposedly continuous variables, which are not usually taken into account; (III) presents empirical results from a measurement invariance study based on 5,272 participants’ responses to a training satisfaction questionnaire in three different organizations and in two different training methods, taking into account the factor structure of the measured construct and the ordinal nature of the recorded data; and (IV) describes the substantive implications in the area of training satisfaction evaluation, such as the usefulness of the training satisfaction questionnaire to measure satisfaction in different organizations and different training methods. It also discusses further research based on these findings.

Keywords

invariancetraining satisfactionevaluation
Type
Research Article
Information
The Spanish Journal of Psychology , Volume 16 , 2013 , E28
Copyright
Copyright © Universidad Complutense de Madrid and Colegio Oficial de Psicólogos de Madrid 2013 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The present study forms part of the results obtained in research projects SEJ2004-05360/EDUC, funded by the Spanish Ministerio de Educación y Ciencia, P06-HUM-01458, funded by the Andalusian Consejería de Innovación, Ciencia y Empresa (Spain), and PSI2011-29587, funded by the Spanish Ministerio de Ciencia e Innovación.
Data were obtained from collaborations between the Universidad of Sevilla and three public organizations: Diputación de Sevilla (PC), Centro de Formación y Perfeccionamiento del Personal de Administración y Servicios de la Universidad de Sevilla (UT), and Instituto Andaluz del Deporte (RS).
The authors greatly appreciate all the comments received from journal reviewers. We believe that the quality and content of the paper have been substantially improved and clarified as a result.

References

Averns, H., Maraschiello, M., van Melle, E., & Day, A. (2009). Evaluation of a web-based teaching module on examination of the hand. Journal of Rheumatology, 36, 623–627. http://dx.doi.org/10.3899/jrheum.080761 Google Scholar
Barron, T. (1997). Is there an ROI in ROI? Technical and Skills Training, January, 21–26.Google Scholar
Bollen, K. (1989a). A new incremental fit index for general structural models. Sociological Methods and Research, 17, 303–316. http://dx.doi.org/10.1177/0049124189017003004 Google Scholar
Bollen, K. (1989b). Structural equations with latent variables. New York, NY: John Wiley & Sons.Google Scholar
Bollen, K., & Bauldry, S. (2011). Three Cs in measurement models: Causal indicators, composite indicators, and covariates. Psychological Methods, 16, 265–284. http://dx.doi.org/10.1037/a0024448 Google Scholar
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: The Guilford Press.Google Scholar
Byrne, B. (1998). Structural equation modeling with LISREL, PRELIS and SIMPLIS: Basic concepts, applications and programming. Hillsdale, NJ: Erlbaum.Google Scholar
Chacón, S., & Shadish, W. (2008). Validez en evaluación de programas [Validity in program evaluation]. In Anguera, M. T., Chacón, S., & Blanco, A. (Coords.), Evaluación de programas sociales y sanitarios. Un abordaje metodológico [Social and health program evaluation. A methodological approach] (pp. 69–102). Madrid, Spain: Síntesis.Google Scholar
Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher’s corner: Testing measurement invariance of second-order factor models. Structural Equation Modeling, 12, 471–492. http://dx.doi.org/10.1207/s15328007sem1203_7 Google Scholar
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255. http://dx.doi.org/10.1207/S15328007SEM0902_5 Google Scholar
Coenders, G., Saris, W., & Satorra, A. (1997). Alternative approaches to structural equation modeling of ordinal data: A Monte Carlo study. Structural Equation Modeling, 4, 261–282. http://dx.doi.org/10.1080/10705519709540077 Google Scholar
Del Barrio, M. V., Carrasco, M. A., & Holgado, F. P. (2006). Factor structure invariance in the Children’s Big Five Questionnaire. European Journal of Psychological Assessment, 22, 158–167. http://dx.doi.org/10.1027/1015-5759.22.3.158 Google Scholar
Díaz, J. F., & Sánchez-López, M. P. (2004). Composite and preferences scales of Morningness: Reliability and factor invariance in adult and university samples. The Spanish Journal of Psychology, 7, 93–100.Google Scholar
DiStefano, C. (2002). The impact of categorization with confirmatory factor analysis. Structural Equation Modeling, 9, 327–346. http://dx.doi.org/10.1207/S15328007SEM0903_2 Google Scholar
Flora, D. B., Finkel, E. J., & Foshee, V. A. (2003). Higher order factor structure of a self-control test: Evidence from confirmatory factor analysis with polychoric correlations. Educational and Psychological Measurement, 63, 112–127. http://dx.doi.org/10.1177/0013164402239320 Google Scholar
Gillespie, M. A., Denison, D. R., Haaland, S., Smerek, R., & Neale, W. S. (2008). Linking organizational culture and customer satisfaction: Results from two companies in different industries. European Journal of Work and Organizational Psychology, 17, 112–132. http://dx.doi.org/10.1080/13594320701560820 Google Scholar
Guilley, W., & Uhlig, G. (1993). Factor analysis and ordinal data. Education, 114, 258–264.Google Scholar
Hoe, S. L. (2008). Issues and procedures in adopting structural equation modeling technique. Journal of Applied Quantitative Methods, 3, 76–83 Google Scholar
Holgado, F. P., Chacón, S., Barbero, M. I., & Sanduvete, S. (2006). Training satisfaction rating scale. Development of a measurement model using polychoric correlations. European Journal of Psychological Assessment, 22, 268–279. http://dx.doi.org/10.1027/1015-5759.22.4.268 CrossRefGoogle Scholar
Holgado, F. P., Chacón, S., Barbero, M. I., & Vila, E. (2010). Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Quality & Quantity. International Journal of Methodology, 44, 153–166. http://dx.doi.org/10.1007/s11135-008-9190-y Google Scholar
Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409–426. http://dx.doi.org/10.1007/BF02291366 Google Scholar
Jöreskog, K. G. (2001). Analysis of ordinal variables 2: Cross-Sectional Data. Text of the workshop Structural Equation Modeling with LISREL 8.51. Jena, Germany: Friedrich-Schiller-Universität Jena.Google Scholar
Jöreskog, K. G., & Sörbom, D. (1996). PRELIS 2: User’s reference guide. Chicago, CA: Scientific Software International.Google Scholar
Kim, K. H. (2005). The relation among fit indexes, power, and sample size in structural equation modeling. Structural Equation Modeling, 12, 368–390. http://dx.doi.org/10.1207/s15328007sem1203_2 Google Scholar
Kirkpatrick, D. L. (1999). Evaluación de acciones formativas [Training programs evaluation] . Barcelona, Spain: Epise.Google Scholar
Marsh, H. W. (1994). Confirmatory factor analysis models of factorial invariance: A multifaceted approach. Structural Equation Modeling, 1, 5–34. http://dx.doi.org/10.1080/10705519409539960 Google Scholar
Menjares, P. C., Michael, W. B., & Rueda, R. (2000). The development and construct validation of a Spanish version of an academic self-concept scale for middle school Hispanic students from families of low socioeconomic levels. The Spanish Journal of Psychology, 3, 53–62.CrossRefGoogle ScholarPubMed
Messick, S. (1994). Foundations of validity: Meaning and consequences in psychological assessment. European Journal of Psychological Assessment, 10, 1–9.Google Scholar
Millsap, R. E., & Tein, J.–Y. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479–515. http://dx.doi.org/10.1207/S15327906MBR3903_4 Google Scholar
Osterlind, S. J. (1998). Constructing test items: Multiple-choice, constructed-response, performance, and other formats. London, UK: Kluwer Academic Publisher.Google Scholar
Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental design for generalized causal inference. Boston, MA: Houghton-Mifflin.Google Scholar
Thayer, P. (1991). A historical perspective on training. In Goldstein, I. L. & Associates (Eds.), Training and development in organizations (pp. 457–468). San Francisco, CA: Jossey-Bass.Google Scholar
Vanderberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices and recommendations for organizational research. Organizational Research Methods, 3, 4–70. http://dx.doi.org/10.1177/109442810031002 Google Scholar
Welkenhuysen-Gybels, J. (2004). The performance of some observed and unobserved conditional invariance techniques for the detection of differential item functioning. Quality & Quantity. International Journal of Methodology, 38, 681–702. http://dx.doi.org/10.1007/s11135-004-5948-z Google Scholar
Ybema, J. F., Smulders, P. G. W., & Bongers, P. M. (2010). Antecedents and consequences of employee absenteeism: A longitudinal perspective on the role of job satisfaction and burnout. European Journal of Work and Organizational Psychology, 19, 102–124. http://dx.doi.org/10.1080/13594320902793691 Google Scholar
Ying, P., & Fan, X. (2003). Assessing the factor structure invariance of self-concept measurement across ethnic and gender groups: Findings from a national sample. Educational and Psychological Measurement, 63, 296–318.Google Scholar