aula Evaluation

aula was tested in four pilot schools in the 2016/17 and 2017/18 school years. You can read about the most important things we learned in our summary report Click here to download the detailed evaluation (report, codebook and analysis) in German as a .zip archive.

For English summary, scroll down.

 


INTRUDUCTION


aula, a unique way for students to make a difference in the lives of their schools was tested in four secondary schools in different regions of Germany from 2016 – 2018. Students, teachers and school leaders took part in a range of workshops and assisted throughout the integration process. Alongside this, the process and teaching resources were continuously revised and refined. Our most important findings are collected below.

Click here to download the detailed evaluation (report, codebook and analysis) as a .zip archive.

METHOD

Semi-structured evaluation interviews were conducted with a sample of 39 students, teachers and school leaders, discussing their own and others’ observations and evaluated with the help of qualitative analysis. We also evaluated the records automatically collected on the aula platform and invited teachers to complete questionnaires on the effectiveness of learning about democracy.


RESULTS

I. Usage of the aula platform

Of the 3205 wild ideas submitted, 20 different projects were developed across the four schools and implemented following successful votes. Active users spent an average of one hour a week using aula.

 

Users accessed aula at home as well as in school. Over the course of the year there were periods of more intensive usage around concrete, well-advertised projects.

II.  Opportunities and advantages

1. Developing self-efficacy

Respondents felt that the greatest benefits of using aula came in the domain of self-efficacy. According to 72% of students, they “feel more strongly that they can change things.” Over half of teachers reported believing that their students’ self-efficacy had increased through aula. In addition, 63% of all respondents reported that their students had become more independent by using aula. Students took on a lot of responsibility for the participation process, and students administrators were appointed in two schools. It was also noted that quieter or less involved students were more forceful in pushing their ideas than previously.

2. Promoting participation through an integrated digital platform

Most participants felt that the online platform made organising collaborative decision-making easier. Alongside the clear structure and transparency of the process, the most frequently noted advantages were:

  • Easier to identify ideas enjoying majorities of popular support
  • Collaborative decision-making was quicker and more accessible
  • Students came into contact with peers with whom they otherwise had little to do.

3. Promoting democratic competencies

Among all respondents the practical experience of democracy was the third most widely noted advantage. Teachers saw the most positive aspect of working with aula as an increase in understanding how democracy works. In addition, teachers observed with significant frequency that aula develops the following skills (cf Himmelmann, 2002):

  • Expressing one’s own opinion (demands, interests, feelings, values) coherently, justifying them clearly and explicitly (lobbying, self-efficacy)
  • Arguing (and standing up) for or against a position according to criteria they set themselves and by others (opinions, criticism)
  • Emphasising collective responsibility, fair norms, as well as developing shared interests and needs, promoting collective problem-solving (taking responsibility, sense of community)

We also noted a positive trend in the following skills:

  • Thinking about the future and solving problems
  • Engaging in dialogue / empathy
  • Sensitivity and solidarity
  • Ability to deal with conflict

 

4. Developing other skills

ENGAGING IN DISCUSSION

Most respondents agreed that aula is a good way to practise formulating ideas (65%) and discussion (53%). Suggestions make a significant contribution to this, as their name suggests a constructive approach to phrasing comments. All teacher respondents felt that formulating suggestions was a productive way of introducing constructive discourse via digital media (cf aula Evaluation 2018, p.34 section 4.2.6.2).

Students volunteered a total of 13 criteria they had borne in mind while forumulating suggestions (being constructive, objectivity, grammatical correctness, appropriateness for young readers, etc).

DIGITAL LITERACY AND KNOWLEDGE OF DIGITAL MEDIA

Older students in particular rate their own knowledge about digital media quite highly, and did not see aula as especially innovative (cf aula Evaluation 2018, p.36, 4.2.7.1) Younger students – particularly those in the 5th and 6th grades – stated that aula was the first online platform they had used at all. In addition, 8 students noted that their teachers’ understanding of digital media improved through using aula.

It has not been possible to measure the extent to which aula really improves technical skills and digital literacy within the framework of this evaluation, as it mainly draws on self-evaluation. There are three reasons we believe aula has an effect on students’ abilities in handling digital media: it introduces students to new concepts such as password security and image licensing through in the framework of introductory workshops; it promotes the continued use of digital media; and it provides a framework for reflection. In future it would be wise to test students’ skills and knowledge using other tools.

 

III. CHALLENGES

The problem most often reported by students was forgotten passwords – this will be remedied with the introduction of the updated app in summer 2018. The second most commonly reported issue for teachers as preventing aula being used was inadequate IT provision in schools (availability of functioning devices, Internet instability etc).

A further challenge arose in occasionally patchy usage of the platform. Students attributed this to: a lack of scope for them to make decisions; their teachers’ lack of motivation and support; alternative avenues for them to participate in democratic decision-making; and limited motivation among students in their final year at school.

This evaluation does not consider why teachers do or do not decide to use aula and the question should be considered in a future study.

 

IV. REQUESTS FOR THE FUTURE

65% of respondents fully intend to continue using aula. 13% said they would continue using aula under certain conditions. The most frequent change request was for an app capable of sending messages and which solves the problem of lost or forgotten passwords.

Students and teachers alike expressed a wish for more learning support in school. This covers a range of scenarios, such as more lesson time dedicated to working with aula, more in-service training time for teachers, or suitable free time in the timetable. Students also asked for more support from teachers, both in terms of time and assistance.


CONCLUSION

Our evaluation shows positive outcomes in developing students’ competencies in the areas of understanding democracy and particularly in experiencing self-efficacy. This supports further development this concept of democratic participation and its implementation at other schools, institutions and organisations. That 76% of respondents intend to continue using aula either completely or under certain conditions is further justification for the project’s further development.

That said, it is essential that teachers take their students’ work seriously and allow the process enough time to support it through teaching and reflection. In order to achieve the ideal technical and learning conditions for aula, schools will in future carry out ‘stock takes’ to ensure the resources and requirements for democratic participation are provided for. In addition, the most common complaint – that of forgotten passwords – will be dealt with through a new app, to be made available for free in Autumn 2018.

The evaluation’s conclusions regarding the development of digital competencies are less unambiguous. Other methods will have to be employed to prove the suspected increase in digital competencies.