ASSESSING 21st CENTURY SKILLS
INTRODUCTORY
” You should look at this section if you have only a limited idea of what 21st Century skills are and how to assess them.”
More and more educators are interested in the assessment of 21st century skills. These are also known as transversal skills, transferable skills, intrapersonal skills, soft skills or generic skills and can be defined as those required for life in a knowledge society. The skills most often mentioned include collaboration, communication, creativity, critical thinking and problem solving. Other skills such as information and communication literacy, global understanding and global citizenship are also sometimes regarded as coming under the umbrella of 21st century skills.
There are a number of issues to consider in assessing these skills.
What are they? Unlike concepts such as mathematics, there are no internationally agreed definitions of 21st century skills. A concept such as ‘collaboration’ can be difficult to define. What does it include? What are its sub-components? Where does it intersect with other 21st century skills?
In addition, there is no agreement on levels within 21st century skills. For example, there is general understanding of the difference between grade 3 and grade 8 mathematics but how could different levels in ‘creativity’ be distinguished? Various efforts are ongoing around the world to agree on definitions and progressions but it is likely to be some time before these are finalised.
Even if some 21st century skills can be defined at a national level, reaching international agreement may be highly complex. For example a concept such as ‘critical thinking’ maybe interpreted very differently in different contexts.A further area of debate is around whether 21st century skills can be regarded as independent of content. For example, can the same definition of problem solving in mathematics be used for problem solving in history, or are these different constructs?
Why should they be measured? The design of assessment is driven by its purpose. This may relate to gathering information on student performance to inform teaching practices or to help better target educational policy. For 21st century skills, however, there is uncertainty about the purpose of assessing these. How, and who, would the information gathered help?
How can they be measured? The assessment of skills such as reading and writing has been taking place for a long period of time. Large amounts of research have been done into how different elements can best be assessed, and there is a great deal of empirical evidence to suggest what works and what does not.This is not the case for 21st century skills.
How can they be marked? Standard forms of assessment may not be suitable for the assessment of 21st century skills. For example, a portfolio of work collected over a period of time may be most appropriate in assessing creativity, but then what criteria should be used to mark this, and how can different portfolios be marked in a standardised way?
How can student contributions be separated? Assessment is traditionally based on ascribing scores to individual students, even when the results are not reported at the level of individual students. But for a skill such as team work in which collaborative work is required, is it possible (or even desirable) to try to provide marks to individual students?
To find out more about the challenges of assessing these skills, move to #Intermediate
INTERMEDIATE
” You should look at this section if you already know about challenges of assessing 21st century skills but would like to know about approaches to addressing these. “
21st century skills are of increasing interest to educational stakeholders, but while many agree on their importance, few agree on what they are and on how to measure them.
Measurement of a construct (from temperature to height to mathematics to intelligence)must start by understanding what it is. There is a lot of debate, but no consensus, on what 21st century skills actually are. For example, communication skills could be seen to comprise a range of elements including empathy, listening and speaking fluency. But ‘listening’ is itself highly complex, and hence determining what to measure is extremely difficult.
Even if a construct was to be fully defined and broken down, another major challenge lies in determining levels of achievement, or progression. For example, what are weak skills in critical thinking in comparison to medium or strong skills in critical thinking? Efforts have been made to identify these distinctions such as the Australian Core Skills for Work Professional Development Framework which divides skills such as ‘Create and Innovate’ into the levels ofNovice, Advanced Beginner, Capable, Proficient and Expert.
Beyond definition, there are also challenges in relation to developing appropriate assessment tasks, marking, and reporting results. Addressing these will require a large amount of work over time to test out various approaches.
A number of initiatives are underway to attempt to develop robust measures for some 21st century skills. For example, efforts to measure collaboration have students undertake tasks while measuring their degree of interaction (such as frequency of posts on a message board), video-recording them (to measure, for example, body language and eye contact) or audio recording them to measure the language they use.
One of the biggest challenges in measuring 21st century skills relates to the need to re-think traditional modes of assessment. Multiple choice items, short response items and long response items are common methods of assessment but are not well suited to the assessment of 21st century skills. This indicates the need to explore new assessment paradigms and there is as yet untapped potential in possibilities such as responding to situations in a virtual reality world.
More practical solutions include the collection of portfolios and group work. While these lend themselves to school and classroom based assessment, scalability remains a major challenge, and they are yet to be attempted at a large scale. One recent attempt to measure 21st century skills at a large scale is the Programme for International Student Assessment (PISA), which included global competence in 2018. Global competence is defined as incorporating:
- Understanding and appreciating the perspective and world views of others;
- Engaging in open, appropriate and effective interactions across cultures;
- Taking action for collective well-being and sustainable development; and
- Examining local, global and intercultural issues.
This was assessed through having students respond to scenarios which promoted them to demonstrate their background knowledge and cognitive skills in navigating global and intercultural issues. Results have not yet been released and it remains to be seen whether or not this proves to be a valid approach to measuring global competence among students, but it is a good example of efforts to measure 21st century skills.
To find out more about efforts to measure 21st century skills, move to #Advanced
ADVANCED
” You should look at this section if you already know about efforts to measure 21st century skills and would like to know more details. “
21st century skills are increasingly of interest to education systems and have begun to be fully integrated into learning and teaching. Examples include: Singapore, which reformed its curriculum in 2010 to focus on teaching and learning 21st Century Competencies (21CC):Australia, which has an integrated curriculum that includes ICT, critical and creative thinking, personal and social capability, ethical understanding and intercultural understanding; and Bhutan which has integrated 21st century skills as “Student Attributes” in their EducationBlueprint 2014-2024. Even where 21st century skills are integrated, however, assessing them remains challenging.
A major debate is the extent to which 21st century skills should be taught–and assessed–integrated within subject domains or as stand-alone domains. Moreover, the absence of generally agreed definitions means that designing instruments to assess them is very much a work-in-progress and there are question marks around the capacity of traditional assessment methods to assess complex constructs.
Where assessment instruments have been developed, their validity is often questioned, with questions about the extent to which they are actually measuring the 21st century skill they focus on or something else instead. Another concern is around reliability and this is exacerbated across countries in which construct definitions may be different.
Despite these concerns, some large scale initiatives have set out to measure 21st century skills. The OECD PISA programme assessed creative problem solving (2012),collaborative problem solving (2015) and global competence (2018). The OECD approach has been to collect data from students in both the cognitive test as well as on background questionnaire.
UNESCO’s Global Citizenship Education aims to prepare learners for the challenges of the twenty-first century. Global citizenship is monitored and measured through a set of indicators that help understand the ways in which learners understand global issues and exhibit skills and competencies relevant to 21st century skills. The table shows examples some indicators.
| Student interprets local context as embedded in global context: | Student’s ethical framework incorporates global sensibility: | Student’s lived experience incorporates global orientation: |
| Understands that local actions can carry global consequences. | Places own and others’ obligations toward humanity at the far end of a spectrum extending out from in-group, and with relatively high intensity of effect; | Chooses to engage with curricular and extracurricular activities in ways that incorporate ‘pro-global citizenship’ ethical orientations. |
| Understands that global forces can be expressed locally. | Expresses success and leadership with positive references to global or extra territorial themes and related actions. | Autonomously incorporates implicit/explicit global focus in curricular and extra-curricular activities. |
Source: Mastercard Foundation Scholars Program, Evaluating Global Citizenship.
The debate around assessing 21st century skills will no doubt continue as there are more and more initiatives that claim to do so.
About the guidance levels
