DETERMINING POLICY GOALS AND ISSUES
# Not Relevant For Teachers
INTRODUCTORY
” You should look at this section if you have only a limited idea of why it is important to determine policy goals and issues for large scale assessment.”
While classroom-based and school-based assessment are motivated by particular educational needs, large scale assessment (either public examinations, national assessments of international surveys) are motivated by political needs. This means that policy makers have questions about the education system that they want to have answers for.
To answer these questions, governments (national, state or provincial) make the choice to provide funding to conduct large scale assessment programmes. A large scale assessment is very expensive to implement so governments want a return on their investment. This means that priorities for the assessment programme need to be decided.
It is not a good idea to try and include assessment of all class levels, all subjects, all students and all learning outcomes unless there is an unlimited budget. Also, there is no point in spending money to collect data if that data cannot be used to provide useful information to help educational policy making or practice.
Priorities often come out of the legal requirements of education in a country. For example, a particular focus might be to find out whether learners from disadvantaged backgrounds or regions are achieving at the same level as learners from more advantaged backgrounds or regions.
Sometimes the government may not make its priorities public, especially if these are controversial topics. But nevertheless the priorities need to be decided right at the beginning of an assessment programme and before any assessment activities start.
One of the important points to recognise is that assessment cannot be used to answer all questions that a government may have about an education system. For example, assessing student learning is not a good way of collecting evidence on:
- The quality of teachers or their professional development needs.
- The attendance of students.
- The quality of school infrastructure.
Therefore it is important that assessment is used as just one of a number of ways of collecting information on the education system. To find out more about how to set policy goals and issues for large scale assessment, goto #Intermediate
INTERMEDIATE
“You should look at this section if you already know why it is important to determine policy goals and issues for large scale assessment but are not sure how this can be done. “
It is often assumed that there is only one type of large scale assessment but actually there are lots of different kinds and each one has different purposes. It is important to clarify the purpose before any other assessment activities begin. Here are some examples:
- Public examinations are large scale and tend to be used to rank or categorise students so that the best ones can be selected for the next stage of education. Examples include GCSE’s, ‘A’ Levels, Year 12 or 13 examinations and university entrance examinations.
- Some large scale assessment can be census based–this means that every student in a particular area is assessed. This is normally done to provide information for students, parents and teachers on how each student compares to their peers. Examples include the Scottish National Standardised Assessments and the National Assessment Program for Literacy and Numeracy in Australia.
- Large scale assessments are often sample based surveys in which only a selected number of students, classes or schools participate. These are used to look at overall student achievement and differences between groups, such as boys and girls. If designed correctly, they can also be used to track progress over time. These can beat the national level, such as or the international level, such as PISA or TIMSS and PIRLS.
So how can governments decide what the most suitable approach to large scale assessment is? This always comes back to purpose. Here are some important questions to ask (and, importantly, there are no right answers as this depends on the context):
- Do we want to give every student an individual result or do we want to collect information about the performance of a group of students(e.g. in a class, at a school, in a district or in a state or province)?
- What are the most important grade levels or stages of education to assess students and why?
- Do we need to assess students every year or term? If not, how often should we assess them and why?
- What learning outcomes or subjects are most important to assess and why?
- What are the most important groups to receive results from the assessment and why?
- What level of detail will be needed by those receiving assessment results and why?
The answers to these questions all have important implications for cost. In general, the more that an assessment programme includes (more subjects, more students and more grade levels) the more expensive it will be. Undertaking a detailed analysis of the cost of different options versus the benefit they will deliver is a critically important consideration for governments to make before starting a large scale assessment programme.
To find out more about how to set policy goals and issues for large scale assessment, goto #Advanced
ADVANCED
” You should look at this section if you already know how to determine policy goals and issues for large scale assessment and are interested in more details. “
There are some important technical considerations that need to be decided at the start of an assessment programme. Here are some example questions and example answers, but these are not comprehensive(you can probably think of more questions and more answers):
- What is the main purpose of the assessment programme? System accountability? Student selection? Tracking student progress over time?
- Which students should be included in the assessment programme? All students?Students in particular grade levels?A sample of students?
- What level of reporting should be used? Individual? Class? School? District? State or Province? Country?
- What subjects should be covered? Literacy and numeracy? Language, mathematics and science? Higher order skills? All subjects?
- What is more important: to give each student an equal chance or that accurate information is to be collected from a cohort of students?
- What kind or reports will be required? Simple percentages or scale scores?
- Should the assessment be high stakes for students, or low stakes?
Another really important topic for consideration is what contextual information needs to be collected. It is common to collect contextual information on things like gender and age. But depending on important education contexts, other things may also be important. For example:
- The level of education and occupation of parents is often important when you are interested in the performance of students from different socio-economic groups;
- The first language of students can be important if you think that this could have an impact on student performance;
- The type of pedagogy that teachers use could be important if you are interested in the connection between pedagogy and achievement; and
- The approach to school leadership and management by school principals could be important if you are interested in the relationship between school type and achievement.
About the guidance levels
