Application of Big Data techniques for the detection of factors associated with performance in high efficiency educational centers

• What do your children do outside school? Out-of-school leisure and academic activities associated with school effectiveness Factors linked to effective schools

     Out-of-school habits and school effectiveness

  Although it appears obvious, it is necessary to highlight the importance of pupils doing activities outside school in order to promote school effectiveness. In this regard, the educational work of schools does not finish when pupils leave for the day; it is also essential to ensure that they are immersed in an environment that fosters habits for personal and academic success. Our study highlights certain pupil habits before and after the school day that are essential factors associated with school effectiveness:

  • The distribution of pupils’ duties and leisure and academic activities outside school affects their academic performance. It is recommended that pupils perform these activities after the school day and that, before school, parents focus exclusively on ensuring that their children are properly rested and fed.
    • While in low-effectiveness schools a high proportion of pupils affirm that they engage in certain activities before attending school (family or work duties, sport, reading, surfing or chatting on the Internet, studying or doing homework, watching television or playing videogames, etc.), frequently carrying out activities after school does not harm effectiveness (with the exception of the work duties).
    • Pupils who regularly converse with their parents after school are associated to a greater extent with high effectiveness.

  

     Out-of-school activities

  In addition to all of these habits, pupils also do out-of-school activities that are either academic or purely leisure-related. Despite the evident importance of a diversity of activities to foster the al-round development of children, the latest scientific evidence shows concern about the danger of overstimulation and overactiviti of young people in today’s hectic world. In this respect, our project has identified certain results that could shed some light on this point:

  • Pupils’ participation in an excessive number of out-of-school activities, whether curricular or extracurricular, is a factor that is clearly associated with school ineffectiveness. In high-effectiveness settings, participation in many different kinds of curricular out-of-school activities is moderate.
    • In low-effectiveness settings, a high proportion of pupils attend a significant number of out-of-school, mainly curricular, classes. Schools in which it is customary for pupils to receive out-of-school support in core subjects of the curriculum are generally identified as being of low effectiveness.
    • Participation in non-curricular out-of-school activities is important and recommended, but, in high-effectiveness schools, pupils tend to have a moderate amount of out-of-school activities.

  

     Out-of-school activities offer

  Having said that, there is no doubt about the positive impact on children of doing some out-of-school activities in order for them to develop properly, but overloaded timetables should be avoided. We should not forget that we all need time for ourselves and time to relax. Regarding the most appropriate kind of out-of-school activities to promote the effectiveness of schools and academic success of pupils, our results suggest that, while no leisure activities are necessarily better than others, the schools should offer a wide variety and focus them on educational goals and aims:

  • Effective schools offer a broad range of out-of-school activities, such as those that are indirectly linked to the school curriculum using a game playing approach (gamification).
    • Among its activities, effective schools include science clubs and competitions, chess clubs and school newspapers.
    • Orientating out-of-school activities exclusively towards leisure pursuits (sport, art, ICT, etc.), thereby disregarding activities linked to the curriculum, is a risk factor for school effectiveness.
• Management teams: Autonomy and effective leadership styles Factors linked to effective schools

     Leadership of the management team

  The leadership role exercised by management teams, both in organisational and instructional tasks, is considered to be a key aspect in fostering school and educational quality. Numerous studies confirm this assertion and emphasise the direct association between an appropriate leadership style and school effectiveness. In this regard, the importance attached by the head teacher and management team to using a democratic and participative management style in the school should be highlighted.

  The evidence identified in this study appears to bear this out, showing that, in effective schools, management teams exercise a style of leadership that is orientated towards clear goals, dialogue, support and the assumption of shared responsibilities:

  • Management teams of effective schools direct their decisions towards the goals and curricular profile of the school. They exercise moderate leadership in relation to educational issues and support the initiatives of the teaching staff without needing to coordinate or direct them.
    • Effective schools explicitly define their goals and curricular profile, and the entire educational community shares them and is orientated towards their achievement.
    • While management teams of less effective schools on ocassions exert excessive control in their leadership tasks on curricular and teaching-learning processes, or on professional development, on others, they ignore these issues.
    • Teachers in effective schools generally consider that their management teams are exercising appropriate leadership and are satisfied with their work.

  

     Institutional assessment

  Another of the fundamental roles of management teams is institutional assessment, which needs to have the active participation of different educational stakeholders. In this respect, it is necessary to have formal and carefully planned strategies that provide information aimed exclusively at improvement. The results obtained in this project point to the following key issues:

  • Assessment activities led by the school have a crucial impact on effectiveness, especially with regard to the establishment of common standards of assessment. Schools that do not use instruments for the academic assessment of pupils and fail to monitor the assessment process in order to orientate it towards improvement are associated with low effectiveness.
    • Effective schools assessment towards obtaining formal information about the capabilities and characteristics of pupils and the educational work of teaching staff, and use standardized tests to guide decision-making
    • Occasional or regular use of standardised tests to assess the performance of pupils, guide decision-making or orientate parents is more common in effective schools.
  • Carrying out assessment actions as monitoring and control measures (external assessment, teacher observation, strict control of teacher training, etc.) without involving the various stakeholders in the process is more associated with low effectiveness.

  

     School autonomy

  The last aspect concerning management teams that stood out in the results obtained from this work tend to relate to school autonomy. Experts do not appear to agree on the advantages or disadvantages of school autonomy, and whether it is better or worse for effectiveness to promote schools with greater autonomy. Our results are equally inconclusive and suggest that systems with greater or lesser autonomy can be effective, with highly autonomous settings being particularly notable for their shared assumption of responsibilities among all educational stakeholders:

  • Sole responsibility by the head teacher or management team for organising teachers, pupils, resources and school budget, or issues related to the curriculum, is associated with low effectiveness. An effective management style is characterised by assuming shared responsibilities and encouraging the participation of other educational stakeholders in decision-making.
    • Teachers in effective schools highlight the importance of shared decision-making and the establishment of shared goals.
    • In effective schools, teachers enjoy greater autonomy in the choice of curricular subjects offered by the school.
    • In effective schools, management teams are more autonomous in establishing the school’s general discipline criteria.
• The role of professional development of teachers in school effectiveness Factors linked to effective schools

     Teacher training in liquid society

  Nobody doubts the vital importance of initial and ongoing training in the professional development of teachers. This need is magnified by the arrival of the so-called Liquid Society. In an environment in constant movement, all previous knowledge acquired is relative today and uncertain in the future. In professions associated with knowledge, such as that of teaching, it is therefore essential to ensure and maintain ongoing, substantial and extensive professional development.

  Nowadays, teachers after their initial training are not expected to be the finished article. What is necessary is for them to develop professionally throughout their careers. Consequently, local, regional and national educational authorities are required to make significant efforts in training their teaching staff.

  Indeed, analyses carried out as part of this project have enabled the gathering of quite clear evidence in this regard. It is evident that, in school settings in which teachers exhibit characteristics that are hihgly orientated towards high effectiveness, the mere existence of teaching staff who are not committed to professional development can jeopardise school effectiveness.

  Further, in schools where there is a stable workforce, with proven professional experience and acceptable levels of teacher satisfaction, the level of teacher training, both initial and related to professional development, stands out as a protective factor for school effectiveness.

  

     Main results obtained

  Based on the data analysis, the following results have been obtained regarding teacher training:

  • Lack of commmitment by teachers to professional development and ongoing training is a common feature in schools with low effectiveness, even when there are a number of protective factors for school effectiveness.
  • Regular and voluntary participation by teachers in professional development programmes in specific areas in which they have the greatest interest should be promoted.
    • While effective schools have teachers that are more involved in their ongoing training, no specific subjects that promote school effectiveness to a large extent are identified.
    • Requiring teachers to be part of professional development programmes, however, appears to reduce the positive effect on school effectiveness of the ongoing training itself.
    • The fact that teachers belong to formal or informal teaching networks relating to subjects in which they have a greater interest protects the effectiveness of the school itself.
  • Schools in which a high proportion of teachers have received initial teacher training, whether academic or linked to practice, tend to be associated to a larger extent with school effectiveness.
  • Schools in which certain teachers have doctorate degrees tend to be associated with high effectiveness, even in settings that have a number of ineffectiveness factors.
• Characteristics of the teaching staff in effective schools Factors linked to effective schools

     Job stability and professional experience

  We are all aware of the situation of instability that many teachers, particularly those in temporary positions and those who have recently passed public examinations, experience. The worldwide economic crisis of recent years, whose effects are still being felt, greatly affected this situation, making school workforces more unstable than before the recession.

  Given this situation, we should be critical and ask ourselves how this issue is affecting school effectiveness in order to establish clear evidence on the relationship between effectiveness and the job stability of school staff. In this regard, the results obtained from this project are transparent. Out of all of the factors involved, the stability of teaching staff is most clearly related to high effectiveness:

  • In effective schools, most teaching staff have extensive professional experience, long service with the school and job stability. While teaching staff experience and stability is a fundamental element, it also needs to be accompanied by other factors to ensure school effectiveness.
    • Schools in which there is significant teacher turnover (where a high proportion are usually newcomers in the current school year), or in which most staff only have a few years of professional experience, are associated with low effectiveness.
    • In schools with low effectiveness, it is common to find unstable workforces with a shortage of permanently contracted teachers.

  

     Professional and job satisfaction

  Our study also reveals another factor that is intimately associated with the stability of the workforce, namely teachers’ levels of job and professional satisfaction. The results show that reduced levels of general teacher satisfaction are clearly associated with low effectiveness:

  • Teachers in less effective schools show generally low or very low levels of satisfaction and this is a determining factor that characterises schools with low effectiveness. Factors or practices that protect effectiveness in settings with dissatisfied teachers are not identified.
    • Teachers in effective schools generally affirm that they are satisfied with their schools and professional work. If asked, these teachers advocate their schools as a good place to work.
    • While low levels of satisfaction are a sufficient condition for identifying a school with low effectiveness, high levels of satisfaction by themselves do not ensure school effectiveness; they need to always be accompanied by other factors that protect effectiveness, such as extensive professional development of teachers.
• The role of families in effective schools Factors linked to effective schools

     Parent participation in schools

  Among the factors associated with school effectiveness that have been discussed the most, but which will probably remain one of the main unresolved issues of the education system, is the participation of parents in schools and the education of their children. This participation includes not only one-way communication between the school and parents, but also the effective involvement of parents in the life and development of the school itself.
  To avoid any of the typical discrepancies or disagreements between the parties involved who have common goals but different perspectives, contact between parents and schools needs to be clearly planned and organised. In this regard, our analysis has identified certain evident factors associated with school effectiveness that refer to the role that parents should play in the academic education of their children. First of all, regarding parent-school relationships:

  • In effective schools the participation of parents in the life of the school is planned and structured, and not subject to the spontaneous initiative of teachers or parents.

  

     Parent academic and emotional support

  Another important point is that parental support and participation in the education of children needs to take place not just within the school, but also in the home. Hence the importance of schools promoting to parents the importance of supporting the academic education of their children. The results of this project indeed bear this out:

  • Parental support for children in school activities, particularly emotional support and praising academic achievement and effort, positively affects academic performance.
    • Schools in which a large proportion of pupils receive little emotional support from parents for their academic work are associated with low effectiveness.
    • Parental encouragement of pupil autonomy and responsibility for homework and studies: it is desirable for pupils to have someone who is available to help them with their school work, but this assistance should not be excessive or extensive.
• School effectiveness and its relationship with ICT Factors linked to effective schools

     ICT and school effectiveness

  Much has been said about the intimate relationship between ICT and school effectiveness and the need to move the current technological environment into the classroom. Most of these analyses, however, have been carried out from a theoretical perspective and lack validation by empirical evidence.

  After years of great efforts to equip schools and promote teacher training, we are in a position to carry out an in-depth analysis of the impact of the integration of ICT into education. As can be seen in this study by the National Institute for Educational Assessment (INEE), the empirical evidence appears to show certain contradictions, and the results obtained here do not stray much from this evidence. Although pupils with greater self-effectiveness and confidence in the use of ICT are associated with high effectiveness, we should be careful not to immerse the pupil into an excessively technological environment.

  

     Results obtained from this research

  The results obtained from applied data analysis indicate the following issues:

  • Effective schools have minimal computer equipment (PCs connected to the Internet, acceptable computer/pupil ratios and interactive whiteboards), and do not necessarily have abundant technological resources (laptops, tablets, Wi-Fi available for pupils, etc.). There are numerous low-effectiveness schools that have widely available ICT resources, but do not use them effectively.
  • Widespread use and integration of ICT by teachers as an end in itself is shown to be a significant factor for ineffectiveness. Conversely, ICT use that is controlled, planned and oriented towards the curriculum is shown to be a factor that protects school effectiveness.
    • Effective schools are characterised by a moderate use of ICT in teaching/learning activities. Highly frequent exposure to computers and the Internet in the classroom is typical of low school effectiveness, as is the widespread use of digital technologies to study, communicate or do homework.
    • Excessively limited or non-existent use of ICT in the classroom also appears to be a factor for school ineffectiveness.
    • The use of ICT channelled towards curricular goals, such as Internet searches for information relating to topics being studied or experimentation with virtual simulators of physical or abstract systems, is associated with high effectiveness. Pupils from effective schools affirm that they perform these kinds of activities more frequently in the classroom and for homework.
  • Puplis from effective schools have greater interest and positive orientation towards ICT, believe that they are more autonomous and competent in their use and value ICT more positively as a tool for individual and social learning.
  • Although the widespread use of ICT for leisure activities is common among pupils in effective schools, filling classrooms and homes with technology without controlling and guiding use is associated with low effectiveness. Organised and regulated use of ICT in the classroom and home is associated with high effectiveness, even in settings with characteristics that are highly prone to low effectiveness.
    • It is highly advisable to orientate the use of ICT towards information activities (searching for useful information, reading news, etc.) and social activities (seeking or providing help with academic or leisure difficulties, participating in online discussions on topics of interest, etc.).
    • Although it is not appropriate to completely restrict access, it is advisable for parents and schools to control the use of the Internet, social media and video games to ensure that it is not excessive.

  Given the above, it is important to emphasise that mere integration of ICT in teaching does not represent any improvement, and may even hinder the educational process. ICT should be integrated in a manner that is controlled, planned and orientated towards clear curricular goals.

• Data mining for identifying factors associated with school effectiveness Factors linked to effective schools

     Classical statistics

  Classic statistical techniques (descriptive, correlational and inferential) have been widely used in the educational field to extract valuable information from databases. These techniques help us to organise and summarise information in order to facilitate the drawing of useful conclusions about the educational environment. The databases used, however, were usually of a manageable size in terms of numbers of subjects and variables.

  With the arrival of the digital society, greater amounts of information began to be managed. Large-scale assessmentsusing volumes of information that would have been excessive for previous techniques became widespread. Classic statistics is less effective at identifying patterns when the data is large-scale, resulting in it not being flexible or informative.

     Data mining: Decision trees

  Data mining consists of techniques designed specifically to identify patterns in massive volumes of data. And decision trees, one of the most widespread data mining techniques, seek to predict or explain where the scores of the subjects of a sample will be located in a variable (known as a criterion variable) based on knowledge of the scores obtained in a set of explanatory or predictive variables.

  Decision trees can, therefore, be valuable in the study of school effectiveness when we have databases with thousands of subjects and explanatory variables (as occurs in the PISA assessment). In practice, selection of the criterion variable is based on the teacher, management team or pupil’s belonging to a school with high or low effectiveness, and selection of predictor variables on other process factors likely to be associated with effectiveness. The tree selects the variables that best differentiate between high and low effectiveness subjects, that is to say, the factors associated with effectiveness. By way of example, the following diagram shows an imaginary decision tree for a sample of pupils.

  

  The tree features a number of branches that lead down to the leaves (rectangles) at the bottom. The rounded maroon rectangles are equivalent to explanatory variables, the branches below them indicate the possible scores of the subjects in the previous variable and each leaf shows in which type of school the pupil who belongs to that group is most likely to be. These leaves contain two numbers: the first refers to the total number of pupils who obtained the scores indicated in the explanatory variables and the second the number of pupils who do not belong to the type of school indicated (from among all of the previous ones).

  In the example, there are 50 pupils with low self-esteem and a level of motivation higher than 3.42 points. If a subject obtains these scores, it is more likely for him to belong to a school with low effectiveness. 19 of these 50 pupils, however, do not actually belong to schools with low effectiveness, which is 38% of the total (19/50*100). This prediction, therefore, has an accuracy of 62%.

  The 4,500 pupils with motivation higher than 3.42 and high self-esteem belong to schools with high effectiveness, so this prediction has 100% accuracy. This means that the pupils in the population with high motivation and self-esteem are clearly associated with high effectiveness. The lowest motivations appear to be more clearly associated with low effectiveness, even in the case of pupils with average self-esteem who perform weekly exercise.

  Based on this tree, we could therefore conclude the following:

  • The motivation variable is clearly associated with school effectiveness. High motivation levels are associated with high effectiveness.
  • With high levels of motivation, low self-esteem is a risk factor for school effectiveness.
  • With low levels of motivation, weekly exercise and high self-esteem can protect school effectiveness, although the level of association is slight in this case.

The Project

Main researcher

Scientific publications

Contact

Project funded by the BBVA Foundation within the framework of its Leonardo Grants for Researchers and Cultural Creators (2017 call), (The BBVA Foundation is not responsible for the opinions, comments and contents included in this document).

Disclaimer

Designed and developed with by AlmaZen Design