Research

2019 Pilot Data Summary

1. Our 3-day (4.5 hour) intervention confirmed our assumptions that middle school children achieved adequate to excellent levels of skill in Lua suitable for solving more complex problems. Overall, on the CTT, we found a 7% increase in scores from the beginning of day 1 to the end of day 3. Interestingly, participants with self-reported less prior experience with coding had a smaller gain (+3.5%) whereas those with self-reported more prior experience had a larger gain (+20.3%). Our N is too low for this to be significant, but we will continue to check to see if this difference appears in our next round of data collection. It may be that those with prior coding experience are more primed for our content, and that we need different approaches for those with less prior coding experience.
2. Interviews revealed several important findings that provided initial evidence for the appeal of our approach to our target audience. When asked what games the campers preferred, all games mentioned included some aspect of multiplayer or playing with friends. Combined with the theme that they all normally play Minecraft with others, this suggests our group really enjoyed the social aspect of video games and of our approach to the camp. We also found a clear trend of participants who used Redstone (Minecraft’s version of electricity enabling machines and operational tools) for its functional aspects (to achieve goals for them) as opposed to aesthetics (also a common use of Redstone). Most students reported their programming experience was mostly with block-based languages, but several had used Java, Javascript, or Python. They emphasized their positive views of learning about using the turtles (a key tool in ComputerCraft, like programmable drones) and the physicality of seeing their it carry out their programs in the Minecraft virtual world. Finally, students in the camp generated a wide range of ideas for using automation to solve problems they face in the game, including many that were STEM related (e.g., managing farms, surveying an area, collecting resources).
3. During the camp, we observed a very high level of engagement and signs of interest triggering. For example, most learners at some point set their own goals and worked from provided examples to add functionality. Also, campers expressed disappointment when the camps concluded each day, and asked for instructions on how to continue using ComputerCraft at home (which we provided). One more general observation with potential consequences for computer science education was that we witnessed widespread confusion regarding writing structured code – most students did not indent their code properly, which is perhaps a direct result from most having exclusively learned coding thus far using block based coding.