Hendrik Drachsler

Profile

Friends

Friends of

Blog

Pages

Files

Hendrik Drachsler
519 days ago

The RecSysTEL workshop was a big step forward for the RecSys research in TEL esp. on: 1. Extending the research community on RecSysTEL, 2. Changing the way RecSysTEL research will be conducted in the future.

Regarding 1, we took advantage of the lucky situation that the ECTEL and the ACM RecSys conference were taking place in the same week in Barcelona. A great opportunity to combine both research communities in one workshop. In the end we created a kind of own mini conference with some core people that attended both workshop days and a wider audience from both research communities that attended one particular day. People traveled between the ECTEL location and the ACM RecSys location so we did not only link the people virtually . Furthermore, we had a keynote from Joseph Konstan, Grouplens research and Kris Jack form the Mendeley startup. Joseph keynote talk was highly appreciated by the ECTEL community and really had an impact on the ongoing research in TEL recommenders.
Kris presented the Mendeley reference system and their datasets that they released in cooperation with the our dataTEL dataset challenge. We recorded both talks and will broadcast them soon.

Regarding 2, one special focus of the RecSysTEL workshop was on datasets that can be used for RecSys research. In order to collect relevant TEL related datasets, the first dataTEL challenge was launched as part of the RecSysTEL workshop. In this sub-call of the workshop, research groups were invited to submit existing datasets from TEL applications that can be used for research purposes on recommender systems for TEL. We opened the first ‘dataTEL system marketplace’ at the 1st day of the workshop.

You can find an overview of the presented datasets in a posting of Guenter Beham in the dataTEL group space at TELeurope.eu.
The most pressing topics in this session were the need to find a standardized meta data structure to exchange datasets, how we can deal with privacy and legal protection rights, anonymization of datasets, pre-processing of datasets, and shared evaluation metrics to compare the effects of TEL recommender systems. This very interactive session was a big step for the community and kept the people in a very crowed room without any window until 6:30 pm, whereas the sun was shining outside in beautiful Barcelona!

It became clear that the ultimate goal of the RecSysTEL research is a research infrastructure where researchers can find well documented and version controlled datasets of different research institutes ranging from formal to informal learning applications. Every RecSysTEL research needs to reference a publicly available dataset to make its results repeatable and comparable and to describe its contribution to the improvement of learning.
The current research practice is mainly based on small-scale experiments in which a few learners are asked to rate the relevance of suggested resources in a controlled experiment. Whereas such experiments offer valuable insights into the usefulness and relevancy of recommender systems for learning, stronger conclusions about the validity and generalizability of recommender experiments are needed in order to create a theory of personalization in TEL.

A theory of personalization in TEL needs more verifiable and repeatable experiments that allow the comparisons of results based on datasets that capture learner interactions. A dataset collection / infrastructure could support researchers to create repeatable experiments to gain valid and comprehensive knowledge about how certain recommender system algorithms perform on certain datasets in a particular learning setting.

The impact of the workshop on the research community is already visible on an increasingly amount of comparison studies that are currently conducted by different research units. The current studies are still quite basic as they apply traditional collaborative filtering algorithms on different educational datasets and report the results of these studies. BUT this is the right way we have to follow to gain valid knowledge on the impact of recommender and personalization of learning.

Based on the big success of the workshop we organized a follow-up workshop at the upcoming ARV2011 in March. Again we will have a two-day workshop, again we will have exciting keynote speakers, and again we will have very exiting contributions, so good signs for another outstanding event.

Pressing topics are:

• publicly available data sets for educational systems
• dealing with legal protection rights towards data sets on a European level
• privacy preservation for educational data sets
• methods of effective anonymization of educational data sets
• management and pre-processing procedures for educational data sets
• future scenarios for educational data sets
• impact of educational data sets for learners and teachers
• mash-ups based on educational data sets
• recommender approaches that are based on educational data
• evaluation methodologies and metrics for educational recommender systems

Besides these topics we are planning a 1st dataTEL competition where different research units will have the opportunity to compete with their algorithms on specific educational datasets. Therefore, we are in contact with Shlomo Berkovsky (ICT Centre, Hobart) who organizes the CAMra competition on context aware recommender systems. With the dataset competition we want to attract also people from other research communities like ACM RecSys but also EDM (Educational Data Mining) and other information retrieval communities to work on educational datasets and increase the knowledge base on personalization technologies in TEL.