![]()
7 Revising the BELMA
Following the AGM and the BELMA ceremony this year, we sat down to start revising the BELMA. Further down follows a detailed project plan - if you do not have time to read it, here is what we would like you to do:
Specification for new criteria:
The criteria used so far are the result of an EU-funded project of the 1990s; they had been based on contemporary research, with a somewhat bias towards language learning. This bias resulted in the need to extensive discussions when other products were presented that as such could not fulfill the requirements (maths books for example). The jury was aware of this and therefore devised countermeasures that will not be necessary with the new set of criteria.
Method:
It is the idea to develop the new criteria in a bottom-up process, based mainly on educated views of stakeholders. One of the reasons for that is that we cannot initiate a corresponding generalized research project at this point - and that the award is situated in a community of practice that can be assumed to have clear ideas on the quality of learning materials.
As far as possible, the bottom-up process will be informed by current scholarly results regarding the quality of learning materials as well as by critical thinking concerning e.g. potentially partial views by specific stakeholders. In addition, its results will be reflected in the light of contemporary research as explicitly as possible, since there are very few resources currently devoted to this kind of research.
Stakeholders:
Materials publishers, researchers, teacher trainers, teachers as well as people from the educational administration (ministries) are seen as the prime stakeholders in this project. The reason why we do not include pupils and parents is that they are not the decision makers when a learning material gets adopted.
Due to specific requirements as a consequence of
In a "pilot" round, we plan to have a first version of the survey questions sent to some 10 stakeholders. Their answers will be used to refine the evaluation methods for the survey. The second round will include 30 stakeholders.
Survey:
The survey itself will be divided into two parts.
- Visit the BELMA Registration page and check out the new description of the categories. If you have comments, please write me a short message.
- Please send me a contact list of 5 - 8 people who you consider stakeholders. When revising the criteria, we want to test them on stakeholders using a survey tool. For the first round, we already have 10 people to test the questions on, but then we would like to do a second round with 30 stakeholders. They should cover different subjects and levels from primary to tertiary. These can be:
- People from your editorial team (any subject, any level)
- Teachers and teacher trainers for different subjects and different school systems and levels
- People from the ministries you know, who are involved in curricula development
- Authors you think are particularly aware of quality and methodology
- Please send me this list by January 30th.
Specification for new criteria:
-
Based on the same set of criteria, there should be two versions which can be communicated:
- A clear transparent short version: this can be used by the publisher as a marketing tool and it enables the "normal" user to see why a material is considered prize worthy.
- A more elaborate jury version that enables the jury to methodically differentiate between the products entered to the award competition.
- The new criteria will have to be illustrated by sample spreads/learning objects/screenshots, etc., possibly taken from previous winners.
-
The new criteria will have to cater for all kinds of learning materials. To warrant this, two areas apart from the criteria will also need looking at:
- there has to be a clear definition for the type of products that can be entered
- a possible revision of the categories may be necessary
The criteria used so far are the result of an EU-funded project of the 1990s; they had been based on contemporary research, with a somewhat bias towards language learning. This bias resulted in the need to extensive discussions when other products were presented that as such could not fulfill the requirements (maths books for example). The jury was aware of this and therefore devised countermeasures that will not be necessary with the new set of criteria.
Method:
It is the idea to develop the new criteria in a bottom-up process, based mainly on educated views of stakeholders. One of the reasons for that is that we cannot initiate a corresponding generalized research project at this point - and that the award is situated in a community of practice that can be assumed to have clear ideas on the quality of learning materials.
As far as possible, the bottom-up process will be informed by current scholarly results regarding the quality of learning materials as well as by critical thinking concerning e.g. potentially partial views by specific stakeholders. In addition, its results will be reflected in the light of contemporary research as explicitly as possible, since there are very few resources currently devoted to this kind of research.
Stakeholders:
Materials publishers, researchers, teacher trainers, teachers as well as people from the educational administration (ministries) are seen as the prime stakeholders in this project. The reason why we do not include pupils and parents is that they are not the decision makers when a learning material gets adopted.
Due to specific requirements as a consequence of
- the age group of the learners concerned,
- the school subject
- the country / language / culture,
In a "pilot" round, we plan to have a first version of the survey questions sent to some 10 stakeholders. Their answers will be used to refine the evaluation methods for the survey. The second round will include 30 stakeholders.
Survey:
The survey itself will be divided into two parts.
- Stakeholders are asked to list what they think makes a learning material a good learning material
- They will be asked to bring possible quality (based on the experience of the jury and the old criteria) in a priority sequence.