Value-added in Higher Education
a new methodology for undergraduate programs in Brazil
Keywords:
Value-added Model, Learning outcomes, Quality measuresAbstract
Among the quality indicators released by the Brazilian Higher Education Assessment System (Sinaes), the Indicator “Difference between Observed and Expected Performance” (IDD) has the purpose of measuring the contribution of the course to student achievement during undergraduate programs. The research presented here offers a new methodology for calculating the IDD (Model IDD-VDCF), examining the philosophical and statistical underpinnings of quality measures, focusing on those that capture the value-added as a student achievement growth. The survey included a sample of 30,668 students, from 911 accounting undergraduate programs in Brazil. The insertion of control variables (at the student and at the institution level) reduced the bias of the IDD estimate associated with the student's selection in specific Accounting Sciences courses. The results call attention to the need to consider the students' learning context when one wants to compare the performance between institutions based on standardized tests. The major contribution of this work is the development of a measure that disentangles more fully what the contribution of program is to student learning, and what merely is a reflection of the capacity that a student brought to the program.
References
ALEXANDER, Nicola; JANG, Sung Tae; KANKANE, Shipi (2017). The performance cycle: The association between student achievement and state policies tying together teacher performance, student achievement, and accountability. American Journal of Education, v. 123, n.3, p. 413-446, 2017. https://doi.org/10.1086/691229
ALEXANDER, Nicola; JANG, Sung Tae. Expenditures on the professional development of teachers: The case of Minnesota. Journal of Education Finance, v. 44, n.4, p. 385-404, 2019.
BAILEY, Thomas, and XU, Di. Input-Adjusted Graduation rates and College Accountability: what is known from twenty years of research? Context for Success, 38p., 2012. http://www.hcmstrategists.com/contextforsuccess/papers/LIT_REVIEW.pdf.
BOGOYA, José Daniel; BOGOYA, Johan Manuel. An academic value-added mathematical model for higher education in Colombia. Ingeniería E Investigación, v. 33, n.2, 76-81, 2013.
BRASIL. Lei nº 10.861, de 14 de abril de 2004. Institui O Sistema Nacional de Avaliação da Educação Superior – SINAES e Dá Outras Providências. 2004. Disponível em: http://www.planalto.gov.br/ccivil_03/_ato2004-2006/2004/lei/l10.861.htm.
CANZIANI, Alex et al (org.). Financiamento da Educação Superior no Brasil: impasses e perspectivas. Brasília: Câmara dos Deputados, 2018. (Estudos Estratégicos n.11). Disponível em: https://www2.camara.leg.br/a-camara/estruturaadm/altosestudos/pdf/financiamento-da-educacao-superior-no-brasil-impasses-e-perspectivas Acesso em: 10 set. 2020.
CUNHA, Jesse; MILLER, Trey. Measuring value-added in higher education: Possibilities and limitations in the use of administrative data. Economics of Education Review, v. 42, p. 64-77, 2014.
FERNANDES, Vivian Duarte Couto et al. ENADE: Uma Análise Sobre os Determinantes do Desempenho Acadêmico dos Estudantes de Ciências Contábeis Desde a sua Primeira Edição. In: Encontro Nacional da Anpad, 42, 2018, Curitiba. Anais... Curitiba, 2018.
FERNANDES, Vivian Duarte Couto; MIRANDA, Gilberto José; ALEXANDER, Nicola. Value-added measures in higher education: a historical contextualization of Brazilian experiences. Revista Brasileira de Estudos Pedagógicos, v.101, n. 259, p.691-720, 2020.
HAIR, Joseph et al. Multivariate Data Analysis. 7 Ed. New Jersey: Prentice Hall, 2010.
INEP - INSTITUTO NACIONAL DE ESTUDOS E PESQUISAS EDUCACIONAIS ANÍSIO TEIXEIRA. NOTA TÉCNICA Nº 33/2017/CGCQES/DAES. Brasília: Inep. http://download.inep.gov.br/educacao_superior/enade/notas_tecnicas/2016/nota_tecnica_n33_2017_cgcqes_daes_calculo_idd.pdf, 2017.
____________ENADE. Brasília: Inep, 2018. Disponível em http://portal.inep.gov.br/web/guest/enade. Acesso em 10 ago. 2020.
____________Censo da Educação Superior: Notas Estatísticas 2018. Brasília: Inep, 2019. http://download.inep.gov.br/educacao_superior/censo_superior/documentos/2019/censo_da_educacao_superior_2018-notas_estatisticas.pdf
JOHNES, Jill. University Rankings: What do they really show? Scientometrics, v. 115, n.1, p. 585-606, 2018. https://doi.org/10.1007/s11192-018-2666-1
KIM, Hoonho; LALANCETTE, Diane. Literature Review on The Value-Added Measurement in Higher Education. Paris: OECD. 52p, 2013. Disponível em: https://www.oecd.org/education/skills-beyond-school/Litterature%20Review%20VAM.pdf Acesso em 19/04/2020.
LAROS, Jacob Arie; MARCIANO, João Luiza Pereira. Análise Multinível aplicada aos dados do NELS:88. Estudos em Avaliação Educacional, v.19, n.40, p.263-278, 2008.
LIU, Lydia. Measuring value-added in higher education: conditions and caveats - results from using the Measure of Academic Proficiency and Progress (MAPPTM). Assessment and Evaluation in Higher Education, v.36, n.1, p. 81-94, 2011.
LIU, Lydia. Value-added assessment in higher education: a comparison of two methods. Higher Education, v.61, p. 445-461, 2011a.
MELGUIZO, Tatiana et al. The Methodological Challenges of Measuring Student Learning, Degree Attainment, and Early Labor Market Outcomes in Higher Education. Journal of Research on Educational Effectiveness, v. 27, n.2, p. 424-448, 2017.
MILLA, Joniada; SAN MARTÍN, Ernesto; BELLEGEM, Sébastien. Higher Education Value Added Using Multiple Outcomes. Journal of Educational Measurement, v. 53, n.3, p. 368-400, 2016.
MITCHELL, Douglas; MITCHELL, Ross. The Political Economy of Education Policy: The Case of Class Size Reduction. Peabody Journal of Education, v. 78, n.4, p. 120-152, 2003. https://doi.org/10.1207/S15327930PJE7804_07
NATIONAL RESEARCH COUNCIL. Getting Value out of Value-Added: Report of a Workshop. Washington, DC: The National Academies Press, 2010.
PIKE, Gary. Assessment Measures: Considerations when Using Value-Added Models in Higher Education Assessment. Assessment Update, v.28, n.5, p.8-10, 2016.
SHAVELSON, Richard et al. On the Practices and challenges of measuring higher education value-added: the case of Colombia. Assessment and Evaluation in Higher Education, v.41, n.5, p. 695-720, 2016.
STEEDLE, Jeffrey T. Selecting value-added models for post-secondary institutional assessment. Assessment and Evaluation in Higher Education, v.37, n.6, p. 637-652, 2012.
USHER, Alex; MEDOW, Jon. A global survey of university rankings and league tables. In: Kehm, B. M.; Stensaker, B. (2009). University rankings, diversity, and the new landscape of higher education. Rotterdam: Sense Publishers, p. 3-18, 2009. https://doi.org/10.1163/9789087908164_002
YEH, Stuart. Understanding and addressing the achievement gap through individualized instruction and formative assessment, Assessment in Education: Principles, Policy and Practice, v.17, n.2, p. 169-182, 2010. https://doi.org/10.1080/09695941003694466
YEH, Stuart. Educational Accountability, Value-Added Modeling, and the Origin of the Achievement Gap. Education and Urban Society. v. 52, n. 8, p. 1181-1203, 2020. https://doi.org/10.1177/0013124519896823
Downloads
Published
Issue
Section
License
Copyright (c) 2021 Avaliação: Revista da Avaliação da Educação Superior
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
A remessa de originais implica a autorização para publicação e disponibilização on-line sem o pagamento de direitos atorais.
A reprodução de artigos publicados na Avaliação só é permitida com indicação da fonte. Utilização de material publicado para fins comerciais é proibida.
