Toward effective learning of cybersecurity: new curriculum agenda and learning methods

Abstract

A recent survey among high school teachers and students in the EU revealed that cybersecurity topics are not included in the programs addressing computer science topics of European schools and that the teachers’ knowledge is not being upgraded regularly for this field. Based on an exhaustive study in 2021–2023 involving high schools in the EU a definition of the educational content and the aligned methodology for high school levels was developed for two areas: cyber-safety and cybersecurity. Available sources, including video platforms, quizzes, presentations, educational games, and hands-on sources for practical exercises, were evaluated and included in the developed fiches with the educational content. The applicability of the developed educational model and the content prepared for upgrading the high school teachers and their student’s knowledge was applied in nine European schools during the 2022–2023 school year. This paper describes the study, briefly presents the educational content, the class preparation and the assessment of the achieved learning objectives obtained with the surveys, interviews, and the Capture the Flag contest. The paper ends with an evaluation of the developed methodology and the usefulness of the approach. Recommendations for future use and development are included in the last paper section.

Introduction

According to the 2022 Eurodyce report on “Informatics Education at School in Europe” (Eurodyce, 2022) informatics is still treated as a new discipline in European high school educational programs. In addition, cybersecurity topics are not included in the curricula for informatics, so they are missing from most of the high school educational programs in the EU. Studies have shown [1] that teachers of informatics topics have never studied cybersecurity topics as a separate discipline, or as a part of the computer science subject, although the importance of cybersecurity for society’s digitalization and sustainable development is widely recognized as being very important. A way to build an educational ecosystem with relevant topics like cybersecurity that reaches target audiences is not yet included in the current European actions for building cybersecurity skills among the Member States’ citizens [2] as the cybersecurity subject is normally only offered within higher education institutions or in courses offered by the professional education market [3].

Today’s young population is changing its internet habits very quickly, in response to considerable societal, market, and technological innovation. They are frequently engaged with online videos, music, gaming, and messaging. In general, their internet use is broadly positive, contributing to the building of digital skills. When enjoying the benefits of the internet, it is important to be aware of the potential risks and to apply safety precautions as young people now have internet access at a very early age. Studies claim that cybersecurity education should be applied in schools because there are many benefits if a school can offer cybersecurity education [4] since the adult participants of seminars organized on the market are reluctant to spend time and money. Schools should provide programs with cybersecurity topics to the larger community as cybersecurity education is changing the minds of individuals and is contributing to a safer digital environment. According to Amankwa [4], schools in the USA are provided with grants and financial allocations from the federal government to ensure support for implementing cybersecurity education. In addition, camps for high school students dedicated to cybersecurity topics are regularly organized each year [5]. One of the examples is the GenCyber [6] summer camp organized for teachers and students within an NSA/NSF support program. However, a study based on a systematic literature review of more than 20 very relevant papers reports that a well-designed curriculum in cybersecurity education at the K-12 level could successfully motivate students at this level to pursue cybersecurity pathways and careers, but today such a curriculum is still missing [7].

Children start using the internet at an early age, and more than 90% of young Europeans are online every day. The use of the internet by teenagers brings many benefits linked to information and communication, but it comes with risks too, including privacy violation, identity theft, ransomware, and the fraudulent use of debit cards, etc. It is, therefore, of paramount importance that school education includes cybersecurity topics that will help these new generations acquire skills that make them aware of major online threats and encourage them to adopt appropriate individual and collective behavior to reduce the risks they are exposed to [8–11]. The major issue when introducing cybersecurity topics at the school level is the teachers’ education, which should ensure the transfer of new knowledge to their students and build the necessary skills with modern delivery methods enriched with interactive tools like educational games [12]. The identified gap in skilled cybersecurity professionals all over the world is also due to the lack of appropriate education at the high school level.

Recommendations for addressing cybersecurity topics or curricula examples in the educational programs of European high schools are missing, as was proven by a survey involving high school management in 14 European countries in 2021 [7]. Digital skills, including the cybersecurity skills built through cybersecurity programs, are understood to represent a combination of abilities, knowledge, and experience that enables an individual to complete a task when working in a digital environment and using digital services as well as being safe. The lack of sufficient attention given to acquiring cybersecurity knowledge by European children and young people has a negative impact on the capacity of a modern, digitized society capable of efficiently reacting to the increasing number of cybercrimes [8,9]. Cybersecurity skills are becoming especially important for economic development, as most experts claim that the digital economy’s winners and losers will be determined by who has these skills [10]. One of the papers addressing this issue found that a lack of cybersecurity education is a global problem and that it could take up to 20 years to address the existing skills gap at all levels of education [11]. The key to producing more cybersecurity graduates to reduce the labor shortage is to establish an educational pathway in the educational process with cybersecurity programs. The study of Jin et al. [12] pointed out that students receiving computer education in high schools are eight times more likely to major in a computer degree and become a cybersecurity professional.

The problem addressed in this paper is 2-fold: a lack of appropriate educational content for the EU’s high school students and a lack of teachers with the knowledge and an understanding of cybersecurity topics, content, and skills. In the paper, both problems are addressed with teaching materials and delivery methods in line with the modern pedagogical approach. Acquiring cybersecurity skills requires an ICT background and knowledge that must be regularly updated to ensure the adoption of technological advances. The demands for new learning methods are now more pressing than ever, especially regarding the need to introduce and upgrade the educational methodology at the high school level. Several studies have found [13–15] that even the education and training in universities in several areas, including ICT, dedicated to preparing high school teachers are still very technocratic, and the educational approach is not promoted in line with the requirements for sustainable education and modern teaching [16].

The major contribution of this paper is (i) the definition of the educational content for the high school level in two areas: and cybersecurity accompanied by available sources composed from video platforms, quizzes, YouTube presentations, educational games, and hands-on sources for practical exercises; (ii) demonstration of the applicability of the developed educational model and content for both audiences, the high school teachers and students from nine European schools during the 2022–2023 school year; and (iii) an evaluation of the learning achievements through surveys and by testing the students’ knowledge with the Capture the Flag game contest.

The paper is organized as follows: The “Related works” section provides a short overview of related previous work on the subject by summarizing the results of an online survey carried out among European high school students and teachers about teaching and learning cybersecurity topics. The survey among other information provides information about the missing contents in the school education dealing with computer science and cybersecurity. The desired educational methods for material delivery of the addressed topics were expressed in the answers of the participating students in the survey. “The survey results and the identified needs” section describes the implementation of curriculum development and the selected learning sources. “The applied methodology and the curriculum design” section presents the applied methodology and the educational content. The implementation and the outcomes are presented in the “Implementing the education and results” section. Class preparation and material delivery are described in the “Class preparation and delivery” section, and the learning outcomes and the student attitudes toward the implemented education program are discussed in the “Learning outcomes and the acceptance of the educational program” section. The “Capture the Flag contest and the students’ results” section informs the reader about the Capture the Flag contest and the student’s results. The “Discussion and assessment of the approach” section discusses the approach applied and the experiences collected. The last section introduces the recommendations and the concluding remarks.

Related works

In this paper, cyber-safety is understood as the means of protecting users from harmful online content. It concerns the emotional and psychological impact of what is seen, read, and heard online. Being cyber-safe means meeting appropriate standards of behavior in the content the users put on the internet, knowing how to avoid harmful interactions online, and being equipped to seek help in case an unexpected result happens.

Cybersecurity is understood as the means of protecting data, systems, and assets on the internet. It refers to the physical and software operation of the networks and computers over which the Internet is delivered. Cybersecurity is described as well as the process, state, or activity in which communication systems and information are protected from modification, unauthorized access, or exploitation [3].

Examples of cybersecurity incidents

Entry of viruses into the system, denial of service by the connected systems, hacked and abused accounts/profiles, man-in-the-middle attacks (an attacker gets access to the communication channel and can see and modify the information exchange), misuse social engineering (where a person tries to acquire the user or company credentials by asking simple questions), performing phishing (getting access to a user’s or company’s credentials in a financial institution and stealing money, encrypting data and requesting payment to unencrypt data and other important information).

Cybersecurity education received additional attention when the problem of a labor force lacking cybersecurity skills was detected almost everywhere. This was estimated to be ∼3 million workers globally, according to cybersecurity workforce studies carried out in 2018 and 2019 [10]. High schools were not part of these studies and proposed programs although some special initiatives addressing girls were launched [8], as well as the Italian national initiative led by the Cybersecurity National Lab from the Italian university alliance offering courses to the Italian high schools during the school year. However, this type of education is not a part of the regular high school education program. Cybersecurity encompasses a broad range of specialty areas and working roles, and this is the reason that no single educational program can cover all the sector-specific knowledge wanted by each audience in the digital society [10]. However, experts agree that there are certain knowledge sets and skills that are essential for most of the population in an individual's everyday life, in participating in the process of studies, or in performing a critical working role they need to adopt [11–13]. Considering the broad range of specialty areas, it is not surprising that cybersecurity education has been addressed differently by various countries that build cybersecurity strategies with different focuses [14,15] of these strategies are formulated as strategies for improving the general state of cybersecurity in the country, which also includes the educational system, but the problem of high school level education is still not sufficiently addressed. Similar findings were reported recently by researchers describing different initiatives for enhancing the safety behavior of young people when using online services [16,17]. Several studies addressing the relevance of cybersecurity education in high schools [18–21] have identified a few topics that should become part of high school education for cybersecurity. Among the listed topics are cyberattacks and vulnerabilities like backdoors, eavesdropping, phishing, and social engineering. Othery studies [22] proposed a slightly different selection of cybersecurity topics to become part of the curriculum in computer science, like data breaches in connection with web-browser configurations, the generation of secure passwords, and the recognition of fake accounts. The most complete overview of cybersecurity topics currently can be found in the UK CyBOK organized base with 19 Key Areas (KAs), each of them supported by two sub-areas. An additional finding of the research that was carried out in the spring of 2022 was the abundance of cyber-safety online platforms that mainly provide advice on how to behave when using online services, explain the danger of certain activities on the internet, and provide information to adults about how to support children and young people in cases that might lead to abuse or when abuse happens. Most of the provided material on the internet is only available in written electronic form (brochures, posters, or presentations, typically in PDF format). No recommendations were found on platforms about how to apply the available educational tools like serious games and videos in cybersecurity educational programs for the high school level. The game-based learning tools available on the internet cover very different topics that include spam, malware, cyberattacks, SQL injection XSS scripting, but they are intended to be used by very large and mixed audiences. Curriculum and fiches development for high school level education requires tremendous teamwork in analyzing the offers and selecting the best solutions that match the training content in the fiches, together with the selected e-sources for the high school education level.

Several researchers who worked in the area and investigated the value of digital game-based learning in cybersecurity education have stressed the need for an improved educational approach by adding more technical and computer-based tools as accessories like quizzes and games [23–25] stressed the cybersecurity game’s potential to enable students to learn in simulated reality and test their skills at solving cybersecurity problems. Evidence from high schools suggests that cognition among students is much greater if they are allowed to engage in classroom activities that support the development of critical thinking and problem-solving, enabled by the use of the mentioned accessories [18]. Gmification has been introduced as a method to motivate students to use a variety of techniques found in popular games, and they were added to the educational modules. Training solutions [12], including games in the area of cybersecurity, have proven their capacity to raise awareness, defense, and effective strategies based on an exhaustive analysis of more than 100 cybersecurity games available on the market.

The survey results and the identified needs

The findings from the research about the missing appropriate cybersecurity education triggered the members of the Concordia Competence Centre to dedicate therwork remedythesituation of missing skills among the EU population by preparing a modular cybersecurity body of knowledge, firstly for workers in the industry. Courses composed of different modules were launched and offered to interested organizations from different economic segments. The modules were shaped in a way that enabled the shaping of an appropriate education process according to the specific industry needs by selecting modules from the offered set. Later, the team working on the cybersecurity curriculum’s development started to explore the situation regarding cybersecurity education among the younger population in the EU and carried out a study to help the selection of KAs that could become part of the EU high schools’ regular education [26]. The team was composed of cybersecurity experts and university teachers, some of them being contributors or editors of the developed methodology and fiches’ content presented below. To extend the applicability of the program to the level of knowledge in a particular high school, three difficulty levels were prepared for each topic, i.e. beginner, intermediate, and advanced. It was decided that the developed curriculum be made up of two major fields, i.e. cyber-safety and cybersecurity, as these areas have different approaches to delivering the content and the learning.

At that time, the UK National Cybersecurity Centre launched the Knowledge Base of KAs in cybersecurity dedicated to training postgraduate degrees in higher education ], that identifies the KAs in all the relevant topics of cybersecurity and the aligned areas such as Human, Organizational, and Regulatory Aspects [24]. The base is licensed under the Open Government License v3.0. However, the KAs and the educational level appropriate for high schools are not yet provided by the CyBOK database.

The intention of the Concordia team to develop a cybersecurity curriculum and training program with modern delivery methods and principles in a high school environment was faced with a lack of information about the current status in EU high schools regarding cybersecurity education. How much students know about cybersecurity when they are online was not as well evident or published. A few research questions in that context required answers before the work dedicated to developing an appropriate curriculum could start [7]. They were RQ1: To what extent are cybersecurity topics included in the EU’s high school education programs? RQ2: What are the missing topics in school programs that provide information for enhancing cybersecurity skills and ensuring safety when students are online? RQ3: How much do high school students know about cybersecurity and self-protection when using the internet? RQ4: What kind of educational delivery methods are most desired by high school students? RQ5: What are the most relevant methods for testing the appropriateness of the developed curriculum and the teaching methodology.

The answers to the research questions were expected to be provided by a survey launched in 2021 all over the EU [25] large pool of stakeholders, namely teachers, students, their parents, and the schools’ managers, were addressed as well. Answers to the research questions were accompanied by the collected information from the interviews and the talks with students’ parents, teachers, and school managers. The survey was prepared on the EU Survey platform in English and launched online with questions translated into several official EU languages, such as German, Spanish, French, Italian, and Greek. The invitation to take part in the survey was disseminated on social media and promoted in specialized high school networks. One hundred percent of the participants provided information about their country of origin, and 97.6% of them declared that they were based in an EU country. One hundred percent of the participants also provided information about which group they belong to high school teacher, student, school manager, or parent. All questions in the survey were answered by the participants by selecting one of the possible answers.

A total of 366 participants provided answers by the end of November 2021, more than half of them were high school students (63% of the sample, the rest of the answers were provided by the participating teachers and parents). Nine EU Member States provided a considerable number of answers to all the survey’s questions, from both audience, students, and teachers. These countries were Romania, Slovenia, Greece, Cyprus, Italy, Spain, Germany, the Netherlands, and Poland. The number of answers received from the other five countries’ high schools was not numerous and was not included in the evaluation of the survey data. The demographic data revealed the almost equal participation of female and male students. The addressed audiences were asked to provide the following information [7]:

• – Demographics (anonymized).

• – Digital services used by high school students in general.

• – Digital services used by high school students in the school environment.

• – Devices used by high school students in general.

• – Devices used by high school students in the school environment.

• – Degree of confidence of high school students during specific online activities.

• – Degree of awareness of high school students regarding online risks.

• – Incidents experienced by high school students related to online risks.

• – Possible subjects that could be discussed within a relevant cybersecurity course for high school students.

• – Type of methods/instruments to be used while teaching cybersecurity at the high school level.

• – Cybersecurity subjects in already-existing courses.

The answers of the participants to the first RQ1 are summarized in Fig. 1.

The data from Fig. 1 reveal that the teachers are informed about the content of the computer science school program that addresses some cybersecurity topics as more than half of the teachers answered with “yes” to the question. Students are also aware that cybersecurity is not taught in their schools. Parents seem to be less informed about what is taught in the school as their answer “I don’t know” shows the highest percentage among the addressed target audiences.

The answers to RQ2 are presented in Table 1.

Table 1 shows that the majority of the students are missing knowledge enabling them to be safe in their online communication with the media platforms. They are also interested in recognizing fake accounts on the web and e-mail services. An average number of students have shown interest in secure downloading (49%) and creating strong passwords. They have shown less interest in avoiding spam and safely sharing files online.

The answers to question 3 provided the data presented in Fig. 2.

e-sources

Figure 2 provides information about how confident are the students when working on the online services. They have given scores to most of the asked questions with scores ranging from 3.26 to 3.97. The lowest scores (2.16–2.49) were given to two questions dealing with online shopping and file management. Students are in general very self-confident that they are capable of protecting themselves when working online. Less interest in online shopping is explicable as they are not high consumers of goods or payable services.

The students’ answers to question QR 4 are provided in Table 2.

The student answers about the delivery methods of cybersecurity topics confirmed previous findings in some studies: They want to have videos, games, and interactive presentations.

The answer to RQ5 was provided by the team that prepared the cybersecurity and cyber-safety education. The teachers who took part in the interviews after the end of the education pointed out that at the start of the process, they had limited knowledge about cybersecurity topics and advanced educational tools, so they had to learn more from the offered courses. The interviews revealed that high school teachers do not upgrade their knowledge regularly in line with the pace of the technology’s development, especially in the area of cybersecurity. The finding was considered in the development of the curriculum and the design of the modules with more advanced knowledge sources, prepared for upgrading the teachers’ knowledge. It was a common understanding that the introduction of new content to the school curriculum should not be complicated and should not require the design of new teaching content, as this was provided in the fiches. An additional challenge was getting the computer labs ready for cybersecurity teaching, as resources and tools were not always known to teachers, and they requested that they be at their disposal and to work immediately. However, it was agreed that the introduction of the new educational content in the school program would improve cyber skills, cyber hygiene, devices, and data protection and would generate ideas about the student’s career prospects. As the survey has shown, the students are familiar with games and seek fun when learning and working, so gamification of the educational content seemed to be obvious [26]. Games offer meaningful and relevant context, which allows learners to understand the situation and the subject more effectively and conveniently. Games, compared with hands-on experiments, are more cost-effective, as they are low risk, provide an individually tailored pace of learning, standardized assessment of the results, and immediate feedback for the learners and the teachers [26]. However, it should be noted here that the design of cybersecurity education is a demanding task, requiring extra work by the teachers, but the teachers should be aware that the involved work does not guarantee that the benefits and the payback for the involved work and time will come. Most benefits are expected to go to the student’s education. Cybersecurity contests and playing the Capture the Flag game were stressed by several researchers to be very helpful tools in helping teachers build cybersecurity skills due to their effectiveness [27,29]. Based on previous works that demonstrated the effectiveness of a cybersecurity lab exercise with the Capture The Flag game, fiches with educational and training material for Capture The Flag were included in the education content.

The applied methodology and the curriculum design

Before starting with the work, e-sources that address education in cybersecurity for students from 14 to 18 years old were inspected. The methodology and KAs were developed in parallel with the selection of the e-sources, which were evaluated for their appropriateness in providing active learning in the classes. Active learning was understood as a provision of interactive lessons supported with interactive videos, games, quizzes, or examples provided with the help of cybersecurity ranges. The collaboration between the high school teachers, the participating experts, and the university professors engaged in teaching cybersecurity and education was continuous and as a result, a set of 12 fiches were prepared for each of the two areas, i.e. cyber-safety and cybersecurity, all of them equipped with information about the difficulty level and the requested students’ prerequisite knowledge. The desired topics identified by the students were included in the selection of online tools, platforms, games, and videos. The flexibility in the application of the education process was assured. The idea was that the education would follow the school’s selection of the module’s content as the survey results revealed that the participating schools differ very much in their programs and attitudes to teaching. The flexibility of the approach enables different needs to be met by a particular school participating in the education exercise and the accommodation of the teaching content to be in line with the student’s knowledge level in that school. The fiche’s content and the e-sources were prepared for the teachers who were teaching informatics, having a medium level of competence in informatics and computer science. The advanced knowledge modules in the fiche collection were prepared to enable the teachers to upgrade their knowledge and, in the case when interest exists among the students, they could elaborate that content with them by offering additional training hours. Class preparation and the selection of the fiche with the appropriate difficulty level were considered the teacher’s task. The students enrolled for cybersecurity education were expected to have basic digital skills, but for the cyber-safety modules, no pre-course knowledge was required. All the fiches contained several existing and already validated e-resources, systems, and tools in both addressed areas.

The significance of the tools in cybersecurity education was pointed out already in a few other studies [30, 31] where the researchers showed that stusdents learnonly 20% of what they hear and read, but can learn 90% of what they have practiced with educational tools. The tools included in the modules address the various objectives, but more attention was paid to the most valuable ones that help the teachers to adapt the content and the teaching by practicing with the exercises on the online testing infrastructures and platforms with facilities that enable hands-on learning and experimenting by solving different tasks. Quizzes and games help the students’ knowledge to be evaluated and advance during the learning. The most popular cybersecurity game Capture The Flag was included in the fiches with three Capture The Flag fiches with different levels of difficulty.

The quizzes encourage the students to test what they have learned. They usually require 15–30 min to complete the task if they consist of one task or are offered multiple choices and questions to be answered. The teachers can select quizzes that provide immediate feedback on the online system and a correct answer in a certain amount of time after the student has answered the question or completed the task. The learning in cases when the taught topic is more complex is enabled by playing cybersecurity games. They are much more common in the set with cybersecurity topics than in the cyber-safety set. Playing games is best applied after some progress is achieved in the learning of the presented content and when the teacher is confident that the level of understanding of the delivered content is satisfactory. The selection of games to be included in the fiches was not an easy task as the number of appropriate serious games with proven learnability [32] for the high school level education of cybersecurity is not numerous [33, 34]. It seems that the cybersecurity area for a less demanding level of knowledge is somehow neglected by game designers. The reason for that comes from the games’ origins, as most of the cybersecurity games were developed within military institutions and are not suitable for a non-military environment, for general education like the one to be implemented in most high schools. However, the games’ inclusion was based on an analysis from previous studies [7].

A summary of the fiches collection is provided in Table 3. A selection of some fiches is presented in Annexes A and B with the Tables defining the student prerequisite knowledge for attending the class of a particular fiche. Fiche A5 and A10, the foundation level are presented in Annex A, and fiches B5 and B10 are presented in Annex B. Other fiches are available in the Concordia report where both authors actively worked in the study and contributed to the report writing [27].

Each fiche listed in Table 3 is equipped with the following information that helps the teachers to prepare the class:

• The level of knowledge addressed by the module.

• The level of knowledge required by the students before they attend the class with that module.

• The learning objectives (LOs) of each fiche.

• The type of knowledge to be conveyed to the students.

• Examples of questions to be discussed with the students during the class.

• Practical examples to be used by the teacher, like games, hands-on exercises and quizzes.

• List of links to e-resources to be used when preparing for the class and later by the students during the class, like games, quizzes, and videos.

Each module is accompanied by a list of URLs for the proposed practical examples, games, quizzes, and suggested additional e-resources. A total of 434 URLs are included in the teaching material, 195 of them address cyber-safety and 239 cybersecurity. They are provided at the end of each fiche. Although all of them were tested before inclusion, regular updating is recommended.

The list of serious games and quizzes in the safety module is modest and is located in fiche A11. The list of included games includes: “Identify and Challenge the information” (topic: Fake News) developed by the University of Portland, “IN ocular Science and Harmony Square.” An example of the recommended quiz is “The all-new NewsWise headline.” Examples from the game list in the cybersecurity modules are: “The Matching Games,” “Map OSI and TCP/IP Layers,” “Cisco learning—Binary Game,” “Print Matching Game,” “Education Arcade,” “Cybersecurity NOVA Labs,” “PBS and Band Runner,” “Solve the Cryptoquote,” “RSA Encryption and Decryption,” “NCSC—the Cyber Sprinter,” “Capture the Flag.” Most of them are available without charge.

Capture the Flag is offered as a specific module composed of three fiches, Foundation, Intermediate, and Advanced. It is recommended to students and teachers interested in the advanced learning of cybersecurity topics. In Annex B, the CTF is represented with an intermediate level fiche.

Implementing the education and results

Class preparation and delivery

Demonstration of the applicability of the developed educational model for both audiences, the high school teachers and students in nine European schools, was implemented in the 2022–2023 school year. The call for participation was sent to the high school’s management that participated in the survey. The endorsement of the school’s participation after they had previously expressed a willingness to participate in the educational process according to the offered educational model required several conditions to be fulfilled by them. The school management agreed to the teaching of the selected curriculum topics becoming part of the ICT classes. Background knowledge in ICT was requested from the high school teachers who applied to take the preliminary courses dedicated to upgrading their knowledge in cybersecurity. The enrolment of the students in the cyber-safety and cybersecurity classes was voluntary, and they were allocated by the school management in groups according to the presented knowledge in the tests, performed before the beginning of the classes. Their purpose was to offer information about the students’ knowledge that helps the fiche with the difficulty level that best matches the students’ knowledge to be selected. The test was recommended to last a maximum of 30 min.

The teachers learned during the organized, online courses, before the education in the schools started, how the proposed methodology works and how it should be applied in the classes. They were asked to test and become familiar with the e-sources and the tools presented in the fiches. A month was given to them to prepare for the first classes before starting their education. For students who showed an advanced knowledge of cybersecurity, the Capture the Flag module was prepared to be delivered to them, together with additional training hours for those interested in the subject and who were planning to participate in the contest by the end of the educational process.

The participating school’s management was asked to ensure classes with an equal participation of male and female students. The teacher’s education started early in October 2022 with a set of online lectures. The engaged instructors who led the courses for the teachers were university professors who contributed to the curriculum’s development. The high school teachers were familiarised with the fiches’ content, how to apply the educational material for the classes, and which fiches were dedicated to upgrading their knowledge of cybersecurity.

The teaching methodology presented to the teachers followed the Dynamic Teaching Model [35] by providing an active learning approach. The involved teacher was considered a person who builds, organizes, provides the content, and cares for the application of the tools in the class. Due to the volatility of some e-resources and their rapid changes, the teachers were asked to check them before the class, and in case they have changed or show a lack of a sufficient and clear explanation of the presented material, ask the team for a replacement. Guidelines for class preparation discussed with the teachers during the teachers’ courses provided opinions and advice that were recommended for use. Special attention was given to the way the fiches were selected when preparing a particular topic for the classes as the topic material was expected to meet the identified knowledge level of the students. The easiest way to find out how much the students know before the teaching starts is to apply a preliminary test with the questions available in each of the fiches. It was suggested when appropriate that the preliminary tests should be replaced with a discussion about the topics that will be presented in a particular class. An open and lively discussion with the students can provide enough information about how much the students are aware of threats when using online services and how much they know about the subject. This method is more suitable for cyber-safety topics as they are, in general, more familiar with them in everyday activities on the internet. In teaching cybersecurity topics, the assessment test is considered almost as a prerequisite for each class portfolio developed from the cybersecurity fiches.

A high degree of interaction between the teachers and the students was very desirable. To increase the focus on one subject and enable, after the materials’ presentation, a lively discussion, all activities related to one topic were recommended to be carried out in one class interval. Among these activities, the preliminary test before the class starts and the storyline introduced by the teacher were considered to be mandatory.

For most of the lessons, the fiches provide practical examples. In presenting them, they can be complemented with some local news relevant to the class topic or with inspirational talks from experts. Such examples are expected to engage the students in the learning process. The bring-your-own-example (BYOE) tactic is also considered to encourage the interchange of opinions between the students and the teacher and to enable more lively discussions in the class. To make the class topic more familiar, the students may be invited to check in advance the lesson to be covered in the next class and to look for cases they would like to share and discuss with the teacher and their peers that fit the class topics. Presenting their examples (BYOE) can happen at the beginning of the class or later, depending on the class topic. The fiches provide information about the learning objectives of each of the presented topics, and when the teacher feels that they have achieved their completion and then a test may be included by the end of the class in the last 10 min. Questions for this type of test are also provided in the fiches, except for the fiches B12, B14, and B15 (Capture the Flag fiches).

The teaching process with the developed curriculum in the schools started in the last week of November 2022. The classes were held in laboratories that were equipped with computers connected to the Internet. The classes started with an introduction about “what a career in cybersecurity means.” Some schools managed to invite experts from the industry who delivered a short speech about that topic by presenting cybersecurity role models and their career experiences, which was concluded with an invitation to the students to consider a career in cybersecurity. In two cases, the speakers were school parents working in the cyber domain, like police, cybersecurity centers, and CERTs. The students were also allowed to bring and use their own devices during the class.

The classes usually started with information about one or two cybersecurity incidents in the area of the fiche topic, presented as a story. The results from the beginning of the class were presented to the students and were discussed with the teacher. The purpose of the discussion was to stress the topic’s priority within the curriculum. Most of the modules offer multiple-choice test questions with different levels of difficulty, which the teacher can select according to the needs of a particular class. Storytelling prepared by the teacher at the beginning of the class is intended to launch the discussion for an easier understanding of the game scenarios to be played during and after the class.

This approach enables better flexibility in teaching and contributes to lively student participation. Another suggested method that increases the students’ participation and interest is the invitation for them to present their examples of cybersecurity incidents that have either been experienced by themselves or learned from elsewhere. This approach triggers the students to check in advance the lesson to be learned in the next class. A look at (personal) cases and discussing the issue with the teacher and peers brings additional benefits and enables smoother acceptance of the subject to be learned. The last 10 teaching minutes before the class ends were suggested to be spent asking the students how they have understood the lesson, either by talking or with a short test to be applied just before the end of the class. The order of presenting different topics was selected by the individual teacher after the evaluation of the student’s knowledge, but alternating the cyber-safety topics with the cybersecurity topics was recommended as a method, based on the findings that cyber-safety topics positively impact the teaching due to the more emotional/psychological content addressed in these topics. The alternating made the learning more relaxed. The topics selected by the teachers were different. Each school accommodates the education according to its ongoing programs. however, the most frequently selected fiches for teaching were "Data Protection (Foundation, Intermediate level),” “Essentials of Cybersecurity (Foundation, Intermediate and Advanced level),” “Attacks, Threats and Vulnerability (Intermediate level),” “Defence against Cyber threats (Foundation level)” and “Capture the Flag (Intermediate and Advanced level).” The classes in all the participating high schools were very well attended. For example, in Slovenia, 365 students from five different high schools attended the classes that were provided by five teachers of computer science. In addition, training hours were applied for the five teams that decided to take part in the contest. The class portfolios in these schools were rich and contained both the foundation fiches as well as the fiches with advanced topics, e.g., “Fake news” was one of the most frequently taught topics. Support was provided by the involved teachers, helping the students to understand the Capture the Flag game mechanics. The additional hours were well attended as they enabled the students to work on the Capture the Flag platforms at their speed.

The selected modules taught in different schools were not the same as the selection was made by the teachers. However, the most frequently selected fiches were Protection of data (Foundation, Intermediate level), Essentials of cybersecurity (Foundation, Intermediate and Advanced level), Attacks, Threats and Vulnerability (Intermediate level), Defence against Cyber threats (Foundation level) and Capture the Flag (Advanced level). These topics and the cybersecurity knowledge adopted in the training hours were considered to be sufficient for taking part in the Capture the Flag contest with an average or low level of difficulty as the students have shown an understanding how the application work.

Learning outcomes and the acceptance of the educational program

After the classes ended at the beginning of March 2023, a survey was carried out in the majority of participating schools to evaluate the adopted knowledge and satisfaction with the learning. Questions were designed following the ACM education recommendation that defines three types of learning outcomes to be assessed in their Computer Science Curricula. They are the following: Familiarity, meaning that a student has understood the concept at the theoretical level, achieved mainly through lectures. Usage: a student understands the concept and can apply it correctly when the situation requires it. Assessment: a student understands the concept and can recognize the given concept in practice [31]. Seven questions were prepared that addressed the suggested learning outcomes, and six questions addressed the satisfaction with the applied education. They are presented in Tables 4 and 5.

The teachers’ opinions were collected from the interviews. Most of them expressed the opinion that the involved modern tools such as game playing are innovative, enable more fun in the classes, and have an easier understanding of the presented material.

A total of 68% of the participants provided answers to the survey’s questions that measured their satisfaction with the teaching. There was no major difference in the answers regarding the gender. The same applies to the teams that took part in the Capture the Flag contest as both genders were present almost equally. Tables 4 and 5 contain the answers from the Slovenian high schools. The answers provided by the students about satisfaction with the classes are presented in Table 4, and the answers about how much the students have learned are presented in Table 5.

The answers in Table 4 clearly show that the students have enjoyed the classes, the presented materials were understandable, and after the end of the education, they feel more confident when working online. The confirmation answers were given by the student majority (from 62% to 74%).

The student’s answers to a set of questions addressing the learning outcomes are confirmative as the majority of them (from 53% to 86%) gave positive answers. One possible explanation of the difference in the student percentage to the answers the two similar questions, e.g. the question from Table 4 “Will you share your knowledge with other students that did not participate in the courses” 19% positive answers, compared to the answers to of the question from Table 5 “Will you share the knowledge you acquired with friends and parents and apply it with them” (45% of positive answers) can be explained with the student’s observation of the classmates as competitors in achieving good scores and reputation in the classes compared to their viewing of the parents and the friends. The last two groups (parents and friends) are considered supportive and friendly companions. However, this explanation should be confirmed with an additional study.

The interviews with the teachers provided some additional insights. The total number of teachers who took part in the conversations about their experience with teaching was 14. They were carried out in the schools of the three countries that applied to participate in the Capture the Flag contest. Most of the teachers were glad to be part of the educational experiment, but most benefits they found came from the content of the fiches and the possibility to learn and prepare classes with material provided on the shelves. The fiches were assessed to be very helpful, as a good way of bringing their knowledge up to date. According to a teacher’s statement, the fiche’s content was found to be very well structured, clearly presented, and with a good red line for implementing the content in a class. The prepared questions to be asked during the classes and those included in the tests were found to be useful and contributed to the discussion with the students. The provided sources for teaching helped them, with less time spent on class preparation, and gave useful guidance in testing the student’s pre-knowledge before the class started. The prepared recommendation for how to use the provided material was assessed to be beneficial. The e-resources, i.e. the available videos, and the game playing introduced more amusement and enjoyment in the classes and made the communications between the students and the teacher easier. The applied active learning enabled better content understanding and easier learning. The teacher’s extra work, in attending the education prepared for them, and the obvious success of the implemented classes were recognized and awarded.

Capture the Flag contest and the students’ results

To verify the results of the applied methodology in more real circumstances an international contest was organized with the game Capture the Flag, which is a game designed to test the cybersecurity ability of the participants. Evidence from other Capture the Flag contests with beginners shows that the students who had typically never worked with a Linux OS or common network and security utilities before the Capture the Flag contest are capable of solving the Capture the Flag contest’s challenges [41] was shown on the European national CTF competition (ECSC) organized each year in October [42].

At the beginning of March 2023, the classes ended, and an evaluation of the acquired knowledge by the students that attended the Capture the Flag classes with the three difficulty levels and the additional training hours. It was prepared as a participation in the international Capture the Flag contest organized as an internal event of the action Teach the Teachers from the Concordia Center program on March 14, 2023. The idea for the development of the Capture The Flag game comes from battle strategy. The participating teams in the Capture the Flag “battle” are supposed to protect their “base” and their “flag.” The other objective of the participating team(s) is to capture the flag from the other competing team(s) without being caught. The adaptation of the battle strategy to cybersecurity allows different scenarios to be prepared, and as a consequence, the results of the contests with different tasks are not the same. This is enabled by the game's nature, which allows it to be played in many different forms [36,37]. The most frequently played form is the “defensive/offensive” [ 31], in which a team defends its network but also attacks the other one. In this type of competition, teams compete to attack and defend servers or services. It is more dynamic and interactive. Each team is provided with its server (or set of services) that they must defend against attacks from other teams. The team’s objective is to ensure the security and proper functioning of their services while simultaneously attempting to exploit vulnerabilities in the services or the servers of other teams on the designed network at the same time. The flag’s format is specific for each competition or contest, and usually, the contest moderator defines the format, the network, and the machine where the scores are collected and registered. The organizer of the contest appoints the moderator, who defines the scoring, and the presentation of the network and provides an explanation of the game’s solutions after the game’s completion [38,39]. Tools like Wireshark that help to analyze the network are usually given to the competitors for their use. The Capture the Flag game was created by the NSA GentCyber program [6] to bring cybersecurity education into the curriculum of students before they get to college. CTF is a very popular game used in many contests organized in 2012 by USA universities within the student’s camps for pre-college cybersecurity education. The game was further developed after its early use, and the number of camps and contests according to some Capture The Flag archives is now more than 100, among them the European annual contest ECSC [40,41]. The reports from the camp contest with K-12 students showed good results and understanding of the security concepts during the camp contest with the Capture The Flag game [42,43] students without previous knowledge of linux OS.

Due to the international dimension of the action Teaching the Teachers and the students from the EU high schools, the Capture the Flag contest runs with the CTF application remotely on the internet as an internal event. All participating schools were invited to send teams, but only three of them answered positively. Each team worked on the computer provided by the participating school. Participation in the contest was limited to a maximum of five high school students in each of the participating teams, but the final number of team members was set to four members. The challenges were prepared according to the Jeopardy Challenges model, based on a sequence of challenges and tasks (questions). This model is easier to organize [37] as the tasks are usually prepared as challenges with different levels of difficulty, enabling the participation of teams with different levels of knowledge and capabilities to solve them. For each of the tasks, a score value was appointed by the moderator ranging from 5 to 20. Five tasks were prepared with different score values depending on the difficulty level applied, e.g. the correct answer to the simplest question provided five points, and the most difficult had 20 points. The scores for each solved challenge were collected on the scoring machine that was controlled by the moderator.

The game was played with 19 teams that were trying to reach the same objective, or the “flag” located in the network server. In the simplest form, the “flags” are usually simple text files placed in a specific location that should be discovered. The playing time and the scores for the solved challenge were defined by the moderator. The contest time was limited to 3 h, this timing is considered as the best time for beginners in a Capture the Flag contest. After the game was completed, the moderator reported the results and presented the correct solutions to the challenges. The times to solve the challenges were recorded, and the team that collected all the flags and got the highest number of scores after the playing time was over was declared the winner of the game. The five challenges that were given to the competing teams from three different countries were formulated as identifying the server’s vulnerabilities (flags) in the established network, answering a set of questions (e.g. the RFC number of a particular IP protocol), exploiting the vulnerability (e.g. enable remote connecting from that server to the network), identify the IP number of the attacker, and find whether a remote shell of the attacker server is working correctly when the server is accessed, find the flag (five-digit number located in a file in the root directory). Appropriate tools were available to teams for scanning the server ports and identifying which ports of the server were open, then finding out the type of protocols running behind these ports. The participants and the vulnerability challenge are presented in Fig. 3. The scoring engine continuously received the solved challenges and confirmed the correctness of the solutions during the allocated playing time. The competing teams did not solve all the presented challenges, so the winner was the team that collected the highest number of scores (87 from possible 100 points). A total of 19 teams of students who attended the classes with the Capture the Flag module came from Croatia, Romania, and Slovenia. The points acquired during the contest ranged from five points (the team with the lowest scores that solved the simplest task) up to 87 points, obtained by the best team, coming from Romania. The ranking of the 19 teams that participated in the contest, each composed of four members, is presented in Table 6 with their names and the country of origin.

Discussion and assessment of the approach

The students who attended the additional training hours were asked after the Capture the Flag classes to report if they understood the delivered content, the rules of the game, and whether they felt capable of solving tasks that simulated real cases. All the participating students gave very positive answers and showed self-confidence regarding their absorbed knowledge. The Capture the Flag game is the most frequently used game for acquiring cybersecurity skills and for assessments of whether the students have understood the cybersecurity concepts that make them capable of acting in reality when they are faced with cyber-attacks [23], which gives an insight into the level of acquired skills and knowledge. The results of the contest held in March 2023 gave positive answers to the questions. Half of the participating teams in the contest received more than 50% of the possible points, and this is considered a good result and evidence that the applied education was useful. However, it should be noted that this contest was the first one where the high school students participated.

The feelings of the involved students about the Capture the Flag exercise were very positive, although they complained about the lack of experiences with Capture the Flag. Students reported as well that they have enjoyed the teaching and the additional spare time spent on training the Capture the Flag. A major part of the satisfaction and pleasure was due the applied interactive teaching and the gaming that gave them a sense of reality. It is worth mentioning that the students attending the Capture the Flag modules showed higher scores in the tests from the regular classes compared to the scores of the other students that did not attend the Capture The Flag teaching module. This confirmed the effectiveness of the Capture the Flag game in acquiring cybersecurity knowledge, as reported by other Capture the Flag events [41–43]. Other students assessed the classes positively after they experienced simulated reality provided by the games. The enjoyment in playing the games was general, and the attraction of the cybersecurity field by some of them was apparent, which is a valuable finding for the producer of a serious game, i.e. to pay more attention to the needs of the young, pre-college population.

The obtained outcomes of the applied methodology were shown to be applicable and useful. The originality of the approach, which has its origin in up-to-date educational methods and material, made the classes attractive. The efforts put into the promotion of the active students’ participation improved the teaching and the students’ learning. The use of some competitive computer games, videos from involved e-platforms, storylines, and other tools are elements that made the classes enjoyable with an interactive participation of the students. The advanced testing of the students’ knowledge before the lessons for accommodating the difficulty level of the delivered material, the provision of the teacher’s active involvement in presenting the content, selected according to the students’ interests and needs, and the additional components applied in the education process made the proposed methodology applicable and attractive. One of the main components that contributed to the applicability of the approach was the offered guidance to teachers for preparing the classes with ready-to-use content, additional learning tools, and available e-sources. The methodology was designed to encourage the teachers to apply specific additional measures to make the classes attractive for the participating students and ensure their participation. The introduction of the BYOE tactic in the area of cybersecurity was another component that made the proposed methodology innovative and acceptable for all involved. The novelty of the developed methodology and its applicability were confirmed by the answers collected during the surveys and the expressed personal interviews of the teachers and the school management that scouted the participants needs in terms of teaching cybersecurity and cyber-safety subjects at a level mostly adopted for the high school students.

However, the main innovative component of the applied methodology is the flexibility that enabled a best match to be achieved between the identified different knowledge levels of the high school students, with their interest in specific cybersecurity topics. Both aspects of computer security field were included in the fiches/modules, providing adequate content as an answer to the different needs in each particular high school that took part in the implementation of the educational model. The cyber-safety content was offered as a supportive and theoretical component of the taught subjects, enabling them good protection from the current threats from the internet threats they facie every day. On the other hand, the cybersecurity content was focused on building more technical skills that help the students to know more about the protection of devices and the internet users themselves. An additional contribution of the presented exercise was the information given to the students about jobs and career paths in cybersecurity that they might decide to follow in the future. Having the ability to practice potentially dangerous techniques in a controlled environment with the games and platforms contributes to the higher students’ confidence and the ability to recognize and defend themselves against malicious acts on the internet, which is the main objective of the cybersecurity education among the youth of Europe.

Concluding remarks

Cybersecurity education at all levels is still an area that needs further support as well as the application of innovative methods and flexible teaching approaches. Cybersecurity is an area with very fast and dynamic development that needs to be followed. Cybersecurity skills have an important role in the digital society as they contribute to the safe usage of the services offered by the digital society elsewhere.

The major contribution of the presented study is in the achieved positive outcomes of the implemented education and the development of a flexible program for teaching cybersecurity at the high school level. The advantage of the presented approach lies in the applied interactive and flexible teaching together with the prepared content for the lessons and the provided links of the e-sources, offerings, videos, quizzes, and games in the area of cybersecurity. The flexibility of the prepared curriculum allows the learning and the teaching to be accommodated easily to the identified knowledge level of the students in a particular school, enabling at the same time upgrading the knowledge of the involved school teachers. The participating high schools from nine EU countries decided to include the developed educational program as a regular subject in each school year.

The integration of cybersecurity and cyber-safety in secondary-school education will have multiple benefits, among them it will enable more young people to pursue a professional cybersecurity career, which will reduce the cyber-skills shortage and the lack of skilled labor. However, it should also be recognized that the introduction of cybersecurity programs in high schools in the EU will still require significant time and effort by all concerned.

Acknowledgements

This paper was prepared and written by the authors listed above, members of the project team Concordia Competence Centre that carried out the presented study. The authors performed the tasks related to the surveys, data analysis, the educational program preparation, its implementation, and the coaching of the teams in the contest CTF. They were active study members and contributors to both reports, the survey and the methodology prepared by the project team. The first author wrote the chapter Cybersecurity and Gaming for Cybersecurity Training, 2021, available online: https://www.concordia-h2020.eu, and the second author is among the four contributors of the Report Teach the Teachers. The authors acknowledge the cooperation and support of the following team members: Felicia Cutas, Romania, Pantelitsa Leondou, Nikos Salamonas, Cyprus, and Ivana Butnic, Germany.

Author contributions

Andrej Jerman Blažič (Conceptualization, Data curation, Formal analysis [supporting], Funding acquisition, Investigation, Methodology [supporting], Project administration, Resources, Software [supporting], Supervision, Validation [lead], Visualization, Writing - original draft [lead], Writing - review & editing, and Borka Jerman Blažič (Conceptualization [lead], Data curation [lead], Formal analysis [lead], Funding acquisition [lead], Investigation [lead], Methodology [lead], Project administration [supporting], Resources [lead], Software [supporting] Validation [lead], Writing - original draft [lead], Writing - review & editing [lead]).

Conflict of interest

The authors declare that no conflict of interest exists.

Funding

The work was supported by the Slovenia Agency for Research, under Grant P2-0037; Internet Society, Slovenian chapter.

Annexes

References