Concept Inventories for Computer Science supported by NSF CCLI Award 0618589, NSF CAREER Award CCF 03-47260, and PITA and GATE grants from the University of Illinois.

New: a recent talk at WCAE overviews our work on this project.

Concept Inventories for Computer Science
  I lead a multi-institution project to develop, validate and disseminate concept inventories for three introductory computer science subjects: discrete math, logic design, and programming fundamentals. This work is joint with Ken Goldman (Wash. U), Cinda Heeren (UIUC), Lisa Kaczmarczyk (U.C. San Diego), and Michael Loui (UIUC) through a joint NSF Course, Curriculum, and Laboratory Improvement (CCLI) grant; the intended work is described by our proposal.

The homepage for this joint project is https://agora.cs.uiuc.edu/display/CSCI/Home.

What is a "concept inventory"?
  The first concept inventory---the Force Concept Inventory (FCI)---was developed by physics educators who identified that students were failing to become Newtonian thinkers (i.e., reason about the world with accepted notions of force and mass) despite of being able to solve rather complex numerical problems. They came to understand that students enter physics courses with deep-seated conceptions (often misconceptions) of how the world works and that significant effort is required to dislodge these misconceptions and replace them with accepted ideas. The Force Concept Inventory was developed to measure the student's progress toward becoming a Newtonian thinker. It consists of a collection of conceptual (non-quantitative) questions that have been selected to test for the presence of common student misconceptions, and as such is a rather reliable measure of a student's conceptual understanding of force. The FCI has played a significant role in the development and adoption of engagement-based approaches to teaching introductory physics at the undergraduate and high-school level. More information about the FCI can be found at David Hestenes's website.

What should concept inventories for intro CS courses cover?
  To guide our work to develop concept inventories, we undertook a Delphi process to solicit expert opinions on topics that are both difficult for students and important. From this effort we identified roughly 12 topics each for introductory courses in discrete math, programming fundamentals, and logic design.

Identifying Important and Difficult Concepts in Introductory Computing Courses using a Delphi Process Ken Goldman, Paul Gross, Cinda Heeren, Geoffrey Herman, Lisa Kaczmarczyk, Michael C. Loui, and Craig Zilles (SIGCSE 2008)
An extended version is available as UIUC technical report UIUCDCS-R-2007-2917

Developing Concept Inventories for Logic Design
  Jointly with Michael Loui (UIUC, ECE) and J.T. Longino (UIUC, ECE MS `06), I have been working to develop a concept inventory for introductory logic design. Logic design is the entry course into computer architecture/engineering, teaching concepts of information representation, combination logic design and optimization, and sequential logic/state machine design and optimization. Through the support of the University of Illinois Provost's Initiative on Teaching Advancement, we have been identifying student misconceptions in these subjects and working to express these misconceptions into a concept inventory-style assessment. This work has been described in the following publications:

Towards CS concept inventories: Assessing learning in Computer Science, Craig Zilles, Birds of a Feather (BOF) Session, Technical Symposium on Computer Science Education (SIGCSE), March 2006.

Student Misconceptions in an Introductory Logic Design Course, J.T. Longino, Michael Loui, and Craig Zilles, American Society for Engineering Education (ASEE) Annual Conference, June 2006.

Boolean Blunders: Identification and Assessment of Student Misconceptions in a Digital Logic Course, J.T. Longino, UIUC M.S. Thesis, July 2006

Boolean Blunders: Identification and Assessment of Student Misconceptions in Digital Logic, J.T. Longino, Michael Loui, and Craig Zilles, 3rd Annual Symposium of the International Society for the Scholarship of Teaching and Learning (ISSOTL), November 2006.