Research Interests

learning analytics, computer-based testing, CS 1, plagiarism detection, computer architecture, compiler construction, dynamic optimization.

Research Projects

The Computer-based Testing Facility and Learning Analytics Hardware/Software Co-designed Run-times for Managed and Dynamic Languages Transactional Memory Efficient Microarchitectures Concept Inventories for Computer Science (CS Education) Master/Slave Speculative Parallelization

Selected recent publications

Distractors Make You Pay Attention: Investigating the Learning Outcomes of Including Distractor Blocks in Parsons Problems (ICER 2024) Evaluating How Novices Utilize Debuggers and Code Execution to Understand Code (ICER 2024) Plagiarism in the Age of Generative AI: Cheating Method Change and Learning Loss in an Intro to CS Course (L@S 2024) Code Generation Based Grading: Evaluating an Auto-grading Mechanism for "Explain-in-Plain-English" Questions (ITiCSE 2024) Quickly Producing "Isomorphic" Exercises: Quantifying the Impact of Programming Question Permutations (ITiCSE 2024) A Comparison of Proctoring Regimens for Computer-Based Computer Science Exams (ITiCSE 2024) Embedded-check: a Code Quality Tool for Automatic Firmware Verification (ITiCSE 2024) Reflections on 10 years of operating a computer-based testing facility: Lessons learned, best practices (ASEE 2024) Comparing the Security of Three Proctoring Regimens for Bring-Your-Own-Device Exams (SIGCSE 2024) One Solution to Addressing Assessment Logistical Problems: An Experience Setting Up and Operating an In-person Testing Center (SIGCSE 2024) Evaluating Mastery-oriented Grading in an Intensive CS1 Course (SIGCSE 2024) Comparing the Impacts of Visually Grouped and Jumbled Distractors on Parsons Problems in CS1 Assessments (CompEd) Code Generation Based Grading: Evaluating an Auto-grading Mechanism for" Explain-in-Plain-English" Questions (arXiv) Evaluating Beacons, the Role of Variables, Tracing, and Abstract Tracing for Teaching Novices to Understand Program Intent (ICER 2023) "I Don't Gamble To Make My Livelihood": Understanding the Incentives For, Needs Of, and Motivations Surrounding Open Educational Resources in Computing (ICER 2023) Am I Wrong, or is the Autograder Wrong? Effects of AI Grading Mistakes on Learning (ICER 2023) Second-Chance Testing as a Means of Reducing Students' Test Anxiety and Improving Outcomes (ASEE 2023) Determining the Best Policies for Second-Chance Tests for STEM Students (ASEE 2023) Comparing Student Outcomes in Online vs. In-person Sections of an On-campus Computer Science Course (ASEE 2023) Investigating the Effects of Testing Frequency on Programming Performance and Students' Behavior (SIGCSE 2023) On Students' Usage of Tracing for Understanding Code (SIGCSE 2023) Using Context-Free Grammars to Scaffold and Automate Feedback in Precise Mathematical Writing (SIGCSE 2023) Discovering, Autogenerating, and Evaluating Distractors for Python Parsons Problems in CS1 (SIGCSE 2023) Reevaluating the relationship between explaining, tracing, and writing skills in CS1 in a replication study (Computer Science Education 2022) On Students Ability to Resolve their own Tracing Errors through Code Execution (SIGCSE 2022) Are We Fair? Quantifying Score Impacts of Computer Science Exams with Randomized Question Pools (SIGCSE 2022) Peer-grading 'Explain in plain English' questions: A Bayesian calibration method for categorical answers (SIGCSE 2022) Students Perceptions and Behavior Related to Second-Chance Testing (FIE 2021) How productive are homework and elective practice? (CSEDM 2021) How should we Explain in plain English? Voices from the Community (ICER 2021) Exploring 'reverse-tracing' Questions as a Means of Assessing the Tracing Skill on Computer-based CS 1 Exams (ICER 2021) Attitudes Surrounding an Imperfect AI Autograder (CHI 2021) Autograding 'Explain in Plain English' questions using NLP (SIGCSE 2021) Pattern Census: A Characterization of Pattern Usage in Early Programming Courses (SIGCSE 2021) Superficial Code-guise: Investigating the Impact of Surface Feature Changes on Students' Programming Question Scores (SIGCSE 2021) Integrating Diverse Learning Tools using the PrairieLearn Platform (SPLICE 2021) Student Perceptions of Fairness and Security in a Versioned Programming Exam (ICER 2020) Comparison of Grade Replacement and Weighted Averages for Second-Chance Exams (ICER 2020) Learning to Cheat: Quantifying Changes in Score Advantage of Unproctored Assessments Over Time (Learning@Scale 2020) A Quantitative Analysis of When Students Choose to Grade Questions on Computerized Exams with Multiple Attempts (Learning@Scale 2020) Strategies for Deploying Unreliable AI Graders in High-Transparency High-Stakes Exams (AIED 2020) A Validated Scoring Rubric for Explain-in-Plain-English Questions (SIGCSE 2020) Measuring the Score Advantage on Asynchronous Exams in an Undergraduate CS Course (SIGCSE 2020) Second-chance testing course policies and student behavior (FIE 2019) Effect of Discrete and Continuous Parameter Variation on Difficulty in Automatic Item Generation (AIED 2019) Predicting the difficulty of automatic item generators on exams from their difficulty on homeworks (Learning@Scale 2019) Reducing Difficulty Variance in Randomized Assessments (ASEE 2019) Every university should have a computer-based testing facility (CSEDU 2019) uAssign: Scalable Interactive Activities for Teaching the Unix Terminal (SIGCSE 2019) Making Testing Less Trying: Lessons Learned from Operating a Computer-Based Testing Facility (FIE 2018) How much randomization is needed to deter collaborative cheating on asynchronous exams? (Learning@Scale 2018) Student and Instructor Experiences with a Computer-Based Testing Facility (EDULEARN 2018) Towards a Model-Free Estimate of the Limits to Student Modeling Accuracy (EDM 2018) Do Performance Trends Suggest Wide-spread Collaborative Cheating on Asynchronous Exams? (Learning@Scale 2017) Investigating Student Plagiarism Patterns and Correlations to Grades (SIGCSE 2017) Modeling Student Scheduling Preferences in a Computer-Based Testing Facility (Learning@Scale, 2016) Bungee Jumps: Accelerating Indirect Branches through HW/SW Co-Design (Micro 2015) Branch Vanguard: Decomposing Branch Functionality into Prediction and Resolution Instructions (ISCA 2015) Computerized Testing: A Vision and Initial Experiences (ASEE 2015) A Psychometric Evaluation of the Digital Logic Concept Inventory (Computer Science Education 2014) Discerning the Dominant Out-of-Order Performance Advantage: Is it Speculation or Dynamism? (ASPLOS 2013) Creating the Digital Logic Concept Inventory (SIGCSE 2010) A Real System Evaluation of Hardware Atomicity for Software Speculation (ASPLOS 2010) Proof by Incomplete Enumeration and Other Logical Misconceptions (ICER 2008) Using Hardware Memory Protection to Build a High-Performance, Strongly Atomic Hybrid Transactional Memory (ISCA 2008) Branch-on-Random (CGO 2008) Fundamental Performance Challenges in Horizontal Fusion of In-Order Cores (HPCA 2008) Identifying Important and Difficult Concepts in Introductory Computing Courses using a Delphi Process (SIGCSE 2008) extended version Hardware Atomicity for Reliable Software Speculation (ISCA 2007) Transactional Memory and the Birthday Paradox (SPAA 2007) Extending Hardware Transactional Memory to Support Non-busy Waiting and Non-transactional Actions (Transact 2006) Probabilistic Counter Updates for Predictor Hysteresis and Stratification (HPCA 2006) A Criticality Analysis of Clustering in Superscalar Processors (Micro 2005) SPIMbot: An Engaging, Problem-based Approach to Teaching Assembly Language Programming (SIGCSE 2005) presentation software Targeted Path Profiling (CGO-2) presentation Master/Slave Speculative Parallelization (Micro-35) presentation Execution-based Prediction Using Speculative Slices (ISCA-2001) Full list of publications

Teaching

CS 105 - Introduction to Computer Science (non-technical majors)   FA19, SP20, FA20 CS 125 - Introduction to Computer Science   FA08, FA13 CS 126 - Software Design Studio   FA16, SP17, FA17 CS 225 - Data Structures   SP19 CS 232 - Computer Architecture II (subsumed by CS 233)   FA03, SP04, FA04, SP05, SP06, SP07, FA07, SP08, FA09, SP09, FA11, SP12 CS 233 - Computer Architecture   FA12, SP13, SP14, FA14, SP15, FA15, SP16, FA16, SP18 CS497/CS598cz - Dynamic Translation and Optimization   FA02, SP03, FA05

Graduate Students

Chinedu Emeka Lucas Flygare Mohammed Hassan David H. Smith

Former Students

Max Fowler Rishi Gulati Binglin Chen Sushmita Azad Saleem Abdulras Lee Baugh Jake Bailey Michael Bond Jeff Cook Brandon Chong Chris Eben David Flint Rahul Joshi Geoffrey Herman Vighnesh Iyer Bhuwan Khattar Edward Lee J. T. Longino Partheesh Mani Daniel McFarlin Shoaib Meenai Andrew Nicholson Naveen Neelakantam Jonathan Pierce James Roberts Nicholas Riley Pierre Salverda Charles Tucker Tianning (Tammy) Xu Changrui Yuan Eric Zimmerman

Student Theses
  Principles of Instruction-Level Distributed Processing Pierre Salverda Ph.D. Thesis, April 2008.
  Efficient User-Mode Exception Handling in x86 Linux Chris Eben, B.S. Thesis, August 2007 (Microsoft)
  Boolean Blunders: Identification and Assessment of Student Misconceptions in a Digital Logic Course J.T. Longino, M.S. Thesis, July 2006 (AMD)
  Profile-directed If-Conversion in Superscalar Microprocessors Eric Zimmerman, M.S. Thesis, July 2005 (Citadel Investment Group)
  Design of an MSSP Verify/Commit Unit Partheesh Mani, M.S. Thesis, August 2005 (Intel Corporation)
  Limited Path Profiling and Correlated Branch Elimination Andrew Nicholson, M.S. Thesis, April 2005 (Amazon.com)
  TraceVis: An Execution Trace Visualization Tool James Roberts, M.S. Thesis, July 2004 (nVIDIA architecture group)
  A Task Optimization Framework for MSSP Rahul Ulhas Joshi, M.S. Thesis, May 2004 (nVIDIA compiler group)
  An Assembler for the MSSP Distiller Eric Zimmerman, B.S. Thesis, May 2004 (continuing for M.S.)
  Program Orienteering Naveen Neelakantam, M.S. Thesis, April 2004 (continuing for Ph.D.)