Heqiao Ivy Wang

selected publications

1. 

   Extending a Pretrained Language Model (BERT) using an Ontological Perspective to Classify Students’ Scientific Expertise Level from Written Responses

   Heqiao Wang, Kevin Haudek, Amanda Manzanares, and 2 more authors

   International Journal of Artificial Intelligence in Education (in review) 2024

   Abs PDF

   The complex and interdisciplinary nature of scientific concepts presents formidable challenges for students in developing their knowledge-in-use skills. The utilization of computerized analysis for evaluating students’ contextualized constructed responses offers a potential avenue for educators to develop personalized and scalable interventions, thus supporting the teaching and learning of science consistent with contemporary calls. While prior research in artificial intelligence has demonstrated the effectiveness of algorithms, including Bidirectional Encoder Representations from Transformers (BERT), in tasks like automated classifications of constructed responses, these efforts have predominantly leaned towards text-level feature, often overlooking the exploration of conceptual ideas embedded in students’ responses from a cognitive perspective. Despite BERT’s performance in downstream tasks, challenges may arise in domain-specific tasks, particularly in establishing knowledge connections between specialized and open domains. These challenges become pronounced in small-scale and imbalanced educational datasets, where the available information for fine-tuning is frequently inadequate to capture task-specific nuances and contextual details. The primary objective of the present study is to investigate the effectiveness of the established industrial standard pretrained language model BERT, when integrated with an ontological framework aligned with our science assessment, in classifying students’ expertise levels in scientific explanation. Our findings indicate that while pretrained language models such as BERT contribute to enhanced performance in language-related tasks within educational contexts, the incorporation of identifying domain-specific terms and extracting and substituting with their associated sibling terms in sentences through ontology-based systems can further improve classification model performance. Further, we qualitatively examined student responses and found that, as expected, the ontology framework identified and substituted key domain specific terms in student responses that led to more accurate predictive scores. The study explores the practical implementation of ontology in classrooms to facilitate formative assessment and formulate instructional strategies.

2. 

   How Students’ Writing Motivation, Teachers’ Personal and Professional Attributes, and Their Writing Instruction Impact Student Writing Achievement: A Two-Level Hierarchical Linear Modeling Study

   Heqiao Wang, and Gary A Troia

   Frontiers in Psychology 2023

   Abs Bib PDF

   Student motivation to write is a pivotal factor influencing their writing achievement. However, individual motivation to write is not independent of the learning environment. It also is crucial for teachers to develop their own efficacy, knowledge, and ability in writing and writing instruction to help them utilize effective instructional methods that stimulate students’ motivation to write and further promote their writing achievement. Given these considerations, we utilized a two-level hierarchical linear model to examine the relationships among student motivation, teacher personal and professional traits, teacher writing instruction, and writing achievement at student and teacher levels. Our analysis of the dataset, which included 346 fourth and fifth graders nested within 41 classrooms, found that motivation had a positive predictive effect on writing ability at both student and teacher levels. Moreover, female students, fifth graders, and typically achieving students demonstrated higher writing achievement than their counterparts. While there were no significant effects of teacher efficacy, knowledge, ability, or professional development on student writing achievement, we observed that higher frequency of classroom management practices during writing instruction had a significant negative effect on student writing achievement. Our full model revealed that the relationship between student motivation and achievement was negatively moderated by teachers’ increased use of instructional practices related to process features and using writing instruction materials, but positively moderated by increased use of varied teaching tactics. Overall, our findings emphasize the importance of contextual factors in understanding the complexity of student writing achievement and draw attention to the need for effective instructional practices to support students’ writing development.

   @article{wang2023students,
     title = {How Students' Writing Motivation, Teachers' Personal and Professional Attributes, and Their Writing Instruction Impact Student Writing Achievement{:} A Two-Level Hierarchical Linear Modeling Study},
     author = {Wang, Heqiao and Troia, Gary A},
     journal = {Frontiers in Psychology},
     volume = {14},
     pages = {1213929},
     year = {2023},
     publisher = {Frontiers},
   }

3. 

   Writing Quality Predictive Modeling: Integrating Register-Related Factors

   Heqiao Wang, and Gary A Troia

   Written Communication 2023

   Abs Bib PDF

   The primary purpose of this study is to investigate the degree to which register knowledge, register-specific motivation, and diverse linguistic features are predictive of human judgment of writing quality in three registers—narrative, informative, and opinion. The secondary purpose is to compare the evaluation metrics of register-partitioned automated writing evaluation models in three conditions: (1) register-related factors alone, (2) linguistic features alone, and (3) the combination of these two. A total of 1006 essays (n = 327, 342, and 337 for informative, narrative, and opinion, respectively) written by 92 fourth- and fifth-graders were examined. A series of hierarchical linear regression analyses controlling for the effects of demographics were conducted to select the most useful features to capture text quality, scored by humans, in the three registers. These features were in turn entered into automated writing evaluation predictive models with tuning of the parameters in a tenfold cross-validation procedure. The average validity coefficients (i.e., quadratic-weighed kappa, Pearson correlation r, standardized mean score difference, score deviation analysis) were computed. The results demonstrate that (1) diverse feature sets are utilized to predict quality in the three registers, and (2) the combination of register-related factors and linguistic features increases the accuracy and validity of all human and automated scoring models, especially for the registers of informative and opinion writing. The findings from this study suggest that students’ register knowledge and register-specific motivation add additional predictive information when evaluating writing quality across registers beyond that afforded by linguistic features of the paper itself, whether using human scoring or automated evaluation. These findings have practical implications for educational practitioners and scholars in that they can help strengthen consideration of register-specific writing skills and cognitive and motivational forces that are essential components of effective writing instruction and assessment.

   @article{wang2023writing,
     title = {Writing Quality Predictive Modeling{:} Integrating Register-Related Factors},
     author = {Wang, Heqiao and Troia, Gary A},
     journal = {Written Communication},
     volume = {40},
     number = {4},
     pages = {1070--1112},
     year = {2023},
     publisher = {SAGE Publications Sage CA{:} Los Angeles, CA},
   }

4. 

   Latent Profiles of Writing-related Skills, Knowledge, and Motivation for Elementary Students and Their Relations to Writing Performance Across Multiple Genres

   Gary A Troia, Heqiao Wang, and Frank R Lawrence

   Contemporary Educational Psychology 2022

   Abs Bib PDF

   Our goal in this study is to expand the limited research on writer profile using the advantageous model-based approach of latent profile analysis and independent tasks to evaluate aspects of individual knowledge, motivation, and cognitive processes that align with Hayes’ (1996) writing framework, which has received empirical support. We address three research questions. First, what latent profile are observed for late elementary writers using measures aligned with an empirically validated model of writing? Second, do student sociodemographic characteristics—namely grade, gender, race, English learner status, and special education status—influence latent profile membership? Third, how does student performance on narrative, opinion, and informative writing tasks, determined by quality of writing, vary by latent profile? A five-profile model had the best fit statistics and classified student writers as Globally Weak, At Risk, Average Motivated, Average Unmotivated, and Globally Proficient. Overall, fifth graders, female students, White students, native English speakers, and students without disabilities had greater odds of being in the Globally Proficient group of writers. For all three genres, other latent profile were significantly inversely related to the average quality of papers written by students who were classified as Globally Proficient; however, the Globally Weak and At Risk writers were not significantly different in their writing quality, and the Average Motivated and Average Unmotivated writers did not significantly differ from each other with respect to quality. These findings indicate upper elementary students exhibit distinct patterns of writing-related strengths and weaknesses that necessitate comprehensive yet differentiated instruction to address skills, knowledge, and motivation to yield desirable outcomes.

   @article{troia2022latent,
     title = {Latent Profiles of Writing-related Skills, Knowledge, and Motivation for Elementary Students and Their Relations to Writing Performance Across Multiple Genres},
     author = {Troia, Gary A and Wang, Heqiao and Lawrence, Frank R},
     journal = {Contemporary Educational Psychology},
     volume = {71},
     pages = {102100},
     year = {2022},
     publisher = {Elsevier},
   }