Kaggle RIIID Knowledge Tracing Competition (21st Place)
In this competition, the challenge was to create algorithms for “Knowledge Tracing,” the modeling of student knowledge over time. The goal was to accurately predict how students will perform on future interactions.
Using data from Riiid’s EdNet, we replicated the state-of-the-art SAINT+ model proposed in Shin, Dongmin, et. al.. SAINT+ is a transformer based architecture that looks at the series of previous answers and questions given to a student using the different encoder and decoder heads to encode different parts of the information.
We augment this architecture with salient aggregate features such as a rolling average of the accuracy of the student until the timestep t, or the number of attempts that the student had on this specific question. These features give additional information to the network, and in some cases propagate the information that is outside the attention window.
As shown in the diagram above (red boxes), we augment the encoder input with tag information and the decoder input with hand-crafted aggregate features.
Using our approach, we finished in 21st position out of 3395 teams.
You can view the full code and details on Github here.
References
Shin, Dongmin, et al. “Saint+: Integrating temporal features for ednet correctness prediction.” LAK21: 11th International Learning Analytics and Knowledge Conference. 2021.