Big Data Student Co-Authors Research Paper in Computing Science Education
Big Data students have a variety of options for completing a co-op work term. One such option is working in a research position with an SFU faculty member. This is what Big Data student Jonathan Bhaskar decided to do. We interviewed Jonathan and his supervisor Dr. Diana Cukierman, Senior Lecturer at the School of Computing Science, to learn more about their exciting research project.
Tell me about your project. What were you working on this summer?
Dr. Cukierman: The project was all about analyzing data around student learning in an introductory programming course. For our study, we used an online tool called “CodeWrite,” which was developed by a group of researchers from the University of Auckland in New Zealand. The group is headed by Paul Denny, who actually served as Jonathan’s unofficial co-supervisor during this project.
I have used CodeWrite in my courses for some years now. CodeWrite allows students to create and solve programming exercises. The system gives students automatic feedback on their submissions and, at the same time, saves a lot of data, such as the time an exercise is first created, the solutions submitted, what exercise a student is working with, etc.
One of the research questions we aimed to explore was whether the activity of students creating their own programming exercises is beneficial to the students’ learning. The course was structured so that half of the students would create exercises, while the other half would answer them. Then, the condition was switched, so that everybody got to invent and solve exercises. This allowed all students to be exposed equally to the tool and also provided data to do the analyses. We also investigated students’ perceptions of inventing exercises via web surveys.
Jonathan, what was your involvement in the project?
Jonathan: The first thing I did was create a ranking of students based on their participation in CodeWrite to create a sense of competition that would motivate students to participate more. When Paul sent me the data from CodeWrite, the information was in many different files, so I took all the files, processed them, and turned them into one huge table with something like 50 columns. This helped Diana and Paul relate data between columns. My ranking was mainly based on two factors: how many exercises a student created or answered, and how difficult these exercises were. The next thing I did was analyze the log file, which had recorded every student interaction with the program. So, what I had to do was look for particular actions, for example, the total time a student spent creating exercises. I looked for the time difference between a student opening the page and a student submitting the exercise. And that was kind of complicated, because the log file contained absolutely everything.
Dr. Cukierman: Yes, each time a student opened a page, for example, that data was logged. A student might have several pages open at once and be working on creating an exercise as well as answering a different one. Jonathan had to process the data in such a way that would allow him to extract the precise information we were looking for.
Jonathan: Yes. And, finally, I created graphs using D3, a JavaScript library, to visualize the results.
What results did you find and what is the significance of the project?
Dr. Cukierman: We have strong support for the idea that inventing programming exercises is a legitimate learning activity leading to better test performance. We also observed that most students in our study believed inventing exercises contributed to their learning in the course.
Jonathan: One of the results was that the top ten students who spent the most time creating questions had an average grade of A- on the assignment, but the top ten who spent the most time answering questions had an average grade of B-.
Dr. Cukierman: Our findings could potentially influence how computing science is taught at the introductory level, where perhaps more emphasis could be placed on students engaging with the material by creating their own questions or exercises.
What are the next steps for the project? Are there any plans for publishing and/or presenting your research?
Dr. Cukierman: We submitted a research paper to Koli Calling, one of the leading international conferences on computing education research, and we are pleased to announce that it was accepted! We will be presenting the paper at the conference, and Jonathan has kindly agreed to create some additional visualisations to show at the presentation, even though he has now completed his co-op term. I’m aIso planning to share the results with students in future courses, since I use CodeWrite in my introductory programming course on a regular basis. I hope that the results will further motivate students to be active participants in their own learning.
Jonathan, what attracted you to this research-based co-op opportunity?
Jonathan: In general, I’m very interested in education, so when I looked at the available research opportunities that were sent to the Big Data cohort, I was particularly interested in this opportunity, because it’s an area where technology intersects with education. So, after that, I met with Diana and we talked about it, and I really liked what she was planning. I think that technology can play a much bigger role in education than it currently does, so I very much enjoyed working on this project.
What are your plans after graduating from the Big Data program and how do you think your co-op experience will aid in achieving your goals?
Jonathan: My goal is to work in the data science field and I will be applying for full-time opportunities in the next few months. What I have done as part of this project is very much data science. Also, being a co-author on a published paper really helps accentuate my skills and accomplishments and will be very useful in my job search.
Dr. Cukierman, why did you want to work with a Big Data student specifically?
Dr. Cukierman: It was a good fit. Here I had all this data and when I got an e-mail from the Big Data Program Director saying there is a possibility to hire a Big Data student for a co-op placement, I thought: “Wow, that would be ideal! These students know how to work with data.” As well, the multidisciplinary area of Computing Science Education is relatively young, so I have to be very creative in how I can get support and research assistance in the area, and this was a great opportunity.
Thank you for the interview, Jonathan and Dr. Cukierman, and congratulations on the publication of your research paper!