Studying philosophical dialogue using Epistemic Network Analysis (ENA) within an international school curriculum

Katarina Hayek

Résumé

Katarina Hayek1 a fait une recherche épistémique dans deux classes de l'École Internationale de Genève dont la finalité est de mesurer l'efficacité épistémique de la Philosophie pour enfants, menée sous la forme de discussions. Son projet est d'utiliser des outils offerts par l'ENA (Epistemic Network Analysis) qui est le fruit de recherches actuelles américaines.

L'intérêt de travailler dans l'école internationale de Genève est de proposer ces outils d'analyse pour aborder des élèves suivant un programme construit par l'école en partenariat avec l'UNESCO. Cette grande institution et plus spécifiquement son Bureau d'éducation, a conçu ce programme dans le but de préparer le jeune à vivre avec les compétences nécessaires au monde du XXIe siècle. Le but de ce programme est que le jeune soit "capable d'interagir en mobilisant et utilisant d'une façon éthique, des données, des connaissances, des aptitudes, des compétences, des valeurs, des attitudes, et des technologies de façon à s'engager effectivement et agir dans des contextes différents du XXIème siècle pour atteindre le bien tant individuel que collectif et global"

Pour analyser l'impact de ce programme sur l'éducation des jeunes, Katarina Hayek a suivi et travaillé des discussions philosophiques avec des enfants (méthode plutôt Lipmanienne) dans un certain nombre de cours, pendant deux ans. Cette analyse a relevé d'une part les attitudes sociales, d'autre part, les interactions intellectuelles, finalement la capacité de penser et s'exprimer librement des participants.

En mettant dans un graphique les diverses habiletés des interactions avec autrui et d'autre part les différentes facettes de la pensée critique, on peut noter les différentes qualités des interactions et de là, l'influence de certaines compétences intellectuelles sur la création d'une interaction sociale.

On peut aussi voir la progression du même graphique dans le temps d'une part d'une compétence et d'autre part de son effet sur une autre et la réciprocité de leurs effets.

Les outils de l'ENA permettent d'analyser finement un état présent d'une classe et des individus qui la composent. Ainsi on peut évaluer un moment présent. Mais cela permet aussi d'évaluer les mêmes données dans un temps postérieur, et ainsi les progrès réalisés.

Dans les graphiques présentés par la recherche de Madame Hayek, on voit des compétences telle la créativité, la question provocante et discutable, le caractère, le respect d'autrui, la gestion de soi, la réflexion, la négociation, l'attention à apprendre, la pensée liée à une attitude de "care", la pensée liée à la critique. Ces compétences sont analysées et mises en relation, quand une compétence tend à enrichir l'autre.

Dans sa conclusion, Madame Hayek montre comment cette analyse permet de mettre en valeur un des aspects du développement du programme de l'Unesco : la progression du développement de compétences essentielles à ce programme grâce à la formation à la discussion philosophique.

Nathalie Frieden

Introduction

Research surrounding the study of classroom discourse is evolving rapidly as researchers become more attune to its contribution to understanding and conceptualising the social contexts of teaching and learning. The Philosophy for Children (P4C) practice is a meaningful, data rich, and collaborative context and can hence serve as a window into the development of students' complex thinking. Yet at present there is less research aimed at closely analysing or assessing the philosophical dialogues themselves as compared to studies looking at P4C's impact in other subject domains. If P4C is to be valued for facilitating meaningful dialogue then it is precisely this dialogue that should be analysed. P4C should be valued as a practice in itself rather than as a means to an evaluative end. It is therefore deemed ineffective to solely measure its effectiveness through external skill-based or subject-based measuring tools (Lien, 2004). Nonetheless, epistemic philosophical progress is difficult to assess and practitioners may often question "are we getting anywhere?"Golding (2017) notes that the best way to know if progress is being made is to get a full view of "the dynamic journey" through studying philosophical dialogue.

I/ Project Overview

This study explores how an innovative tool, Epistemic Network Analysis (ENA), could provide a method to closely study P4C dialogue and further highlight its importance within the primary school curriculum at the International School of Geneva.

A total of 341 minutes of P4C dialogue from four classes (N=87 participants) was analysed from September 2019 to March 2020. The dialogue was transcribed and closely studied in order to code for student demonstration of curriculum competences. Subsequently, ENA software was used to quantify, visualise and analyse these competences in network models.

II/ Research Background

The International School of Geneva's Curriculum: The ULP Framework

In September 2018, the International School of Geneva (ISG) in partnership with UNESCO's International Bureau of Education (IBE) created and launched the Universal Learning Programme (ULP): a K-Year 11 (ages 3-15) curriculum developed in response to the perceived need to provide students with 21st century competences. It is founded upon the belief that competence is "the developmental capacity to interactively mobilize and ethically use information, data, knowledge, skills, values, attitudes, and technology to engage effectively and act across diverse 21st century contexts to attain individual, collective, and global good" (Marope, 2017). From September 2018 to present, the ULP curriculum has been progressively implemented at the International School of Genevaand is currently in the process of undergoing appraisal and evaluation.

III/ Philosophy for Children (P4C) and Assessment

The Philosophy for Children (P4C) practice is a fundamental component of the ISG primary school curriculum and aims to develop both collaboration and character in students. The P4C programme was created by Professor Matthew Lipman and his colleagues in the 1960s and 1970s in New Jersey, USA. However, the roots of P4C developed long before the 1960s. The programme is built upon ancient Greek ideas of Socratic dialogue where philosophical discussion is sparked by asking questions or reflecting upon stories.

At the International School of Geneva, philosophical sessions are often started by reading a novel that stimulates thought and leads students to collaboratively create their own questions. Students are invited to vote on which question they would like to discuss and then there is a group dialogue surrounding the question selected.

According to Vygotsky, the classroom is transformed into a micro-society where there is a simulation of the real ethics of social life (1985). Dewey (1983) noted that this is especially significant when learners interact together in what is referred to as a community of inquiry (CI). Within the CI, it is thought that learners "internalise universal concepts and fundamental principles of social life on a day-by-day basis" and their dialogue becomes an individual and social experience (Dewey, 1983; Vygotsky, 1985 as cited in Daniel &Auriac;, 2011). The objective of P4C is not for students to "learn philosophy" by memorising famous philosophers' names or theories but rather to have them "do philosophy" by engaging in the practice of philosophy through critical thinking.

Within the ISG, P4C provides a setting in which students are encouraged to think for themselves and to practice their skills in context. It focuses on what students are able to express through a facilitative approach. It is also an area where transdisciplinarity is inherent, with topics discussed often branching out into differing areas of knowledge. It therefore provides a potential way to examine multiple linked curriculum competences simultaneously.

A Potential Tool for P4C Assessment: Epistemic Network Analysis

Epistemic network analysis (ENA) is a network modeling method developed by David W. Shaffer and his colleagues in the early 2000s at the Wisconsin Center for Education Research at the University of Wisconsin-Madison, USA (Shaffer et al., 2009).

In order to understand the method of ENA, it is useful to first consider its origins in epistemic frame theory. Stemming from the concept of Communities of Practice (Reimann, 2008), epistemic frame theory suggests that any community of practice has a culture that is composed of skills, knowledge, identity, values, and epistemologies. This collection forms the epistemic frame of the community of practice. In simple terms, ENA is a form of network analysis designed to closely study and assess these epistemic frames (Shaffer, 2018). It attempts to study the patterns of association between knowledge, skills and values as they are expressed in dialogue. ENA bridges principles from social network analysis (SNA) and discourse analysis in order to model patterns in what people say and do (Wooldridge, Carayon, Shaffer, & Eagan, 2018). It was originally developed in order to model theories of cognition, discourse and culture. In contrast with other network modeling techniques, ENA is designed to study a small set of constructs that are demonstrated through highly dynamic and dense interactions (Gasevic&Ferreia;, 2019).

The ENA method is founded on the idea that "the structure of connections among cognitive elements is more important than the mere presence or absence of those elements in isolation" (Shaffer, 2018). The connections in data are therefore viewed as valuable for critical analysis (Shaffer et al., 2009). It builds on the theories of DiSessa (1985), who defined learning as a process where students connect elements of experiential knowledge through their inherent frameworks in order to develop new knowledge and deep understanding. Within education, ENA has been used extensively by researchers. For example, Chiu & Linn used ENA to show how learners develop STEM expertise by "constructing a knowledge web: a repertoire of ideas and the connections among them" (2011). It has been used to model cognitive connections during problem-solving among students and their interaction with mentors (Shaffer & Nash, 2012). ENA has also been used in a variety of research contexts outside of the field of education. Nonetheless, it has not been used to study P4C dialogue and there remains a gap in the research on how it could be effective in studying the thinking patterns within philosophical settings.

IV/ Research Design

The project presented here applied ENA methods to P4C sessions in order to closely examine students' complex thinking during philosophical discussions. This section outlines the research steps and decisions taken.

A/ Sampling strategy and characteristics of the study group

The International School of Geneva's primary school comprises both French and English language classes with students from age three to ten. Students begin to follow P4C lessons at age five. They then follow regular weekly or bimonthly P4C sessions. Classes from grades two to four which conducted these regular P4C sessions in English were included within the study. Grade one was excluded based on the reasoning that they had limited experience of the P4C programme. This resulted in data from four classes (with a total of 87 students) in years two to four to comprise the sample for this study.

B/ Process of video data analysis and ethical considerations

In order to closely analyse the conversations taking place during P4C sessions, video recordings are regularly made by classroom teachers and staff and stored within the school data repository. For the purpose of this study, the video recordings were anonymously transcribed without reference to particular student names or classes. These anonymised transcriptions served as the foundation for subsequent coding analysis.

C/ Creation of the "Codebook"

A document, named the "codebook", outlines each of the school's curriculum constructs. For each construct it provides its definition, guidance notes, keywords and some examples of how the construct may be identified in practice. Table 1 below is an extract example of the codebook.

Table 1: Extract from ISG codebook

These constructs can also be referred to as "codes". Within social sciences, coding is the analytical process of attaching meaningful attributes, "codes", to data (Ingram & Elliott, 2020). For the purposes of this study, the ISG "codes" were compared and refined against classroom data.

The codebook remains a working document. The focus here is not on finding the perfect definition of each construct, but rather a working definition that serves to unite the community in their understanding of how the competences are understood within the context of P4C. Furthermore, as this research takes perspectives from the Grounded Theory approach, the generation of the codebook and the research process as a whole is purposefully iterative. This is described more in the next section.

D/ Coding: The application of the codebook to classroom data

An early adopter of the grounded theory, Charmaz, stated that "codes rely on interaction between researchers and their data. Codes consist of short labels that we construct as we interact with the data." (2012, p. 5). The coding process was therefore largely an iterative and recursive process.

The classroom P4C transcripts were coded line by line for presence or absence of constructs. Coding was undertaken by two researchers with differing perspectives and backgrounds in order to maximise objectivity. The data was coded by an English-speaking native with a background in teaching, and a coder of Chinese background who worked in the field of economics and educational policies. Each coder coded the transcripts and discussed differences. During coding, each utterance was coded for occurrence of constructs, resulting in coded transcriptions. Thus, originally qualitative conversations were operationalised for detailed analytical processes carried out using ENA.

E/ Modelling networks using the epistemic network analysis (ENA) tool

Coded transcriptions were uploaded to the ENA Web Tool version 1.6.0 (Marquart, Hinojosa, Swiecki, & Shaffer, 2018). Within the ENA tool, the coded transcript data is segmented according to discourse analysis principles: it is firstly broken up into lines which represent the smallest unit of data. These lines are then grouped into stanzas (lines which are related to one another). In doing so, ENA adopts three key assumptions. First, that it is possible to systematically identify a set of meaningful features in the data (codes). This assumption further implies that utterances (or lines) can be coded according to the competence codebook. Secondly, it assumes that the data has local structure represented in conversations (or stanzas). Thirdly, the way in which the codes are connected to one another is significant for analysis of the data (Shaffer, Collier, &Ruis;, 2016).

For the purpose of this study, all lines of data (all student and teacher utterances) within each lesson were considered. A moving window process was applied in order to construct a network model for the data. This shows how codes in one line are connected to codes that occur within the recent temporal context (Siebert-Evenstone et al., 2017), which was defined as 10 lines. The ENA model aggregates the resulting networks in the model according to a binary summation where the networks for a given line reflect either the presence or absence of a co-occurrence between codes, after which it normalises the networks before subjecting them to the dimensional reduction. This is an important step as it accounts for the fact that different units of analysis may have different amounts of coded lines in the data. Singular value decomposition (SVD) was applied as the dimensional reduction, as it maximises the variance explained by each dimension (see Shaffer et al., 2016 for a more detailed explanation of the mathematics; see Arastoopour, Swiecki, Chesler, & Shaffer, 2016 and Sullivan et al., 2018 for examples of this kind of analysis). The model ultimately results in a network graph where the node (dots) size represents the frequency of the code occuring and the thickness of the lines connecting these nodes shows how strong the relationship is between those codes. These networks can be used for visual, descriptive and statistical analysis (Wooldridge et al., 2018).

V/ Results

The results presented in this section highlight ENA's capabilities in reflecting the community of inquiry's thinking during philosophical dialogue.

A/ Competence assessment within a single class : Year Two data

In order to illustrate how ENA can be used to study a single P4C session, dialogue from a randomly selected P4C session from a year two class within the school repository was analysed. Table 2 presents an overview of the data.

Table 2: Year Two (21 students, 9 boys, 12 girls, ages: 6-7, 1 female teacher)
P4C session overview

As shown in Table 2, the question for discussion was "Why do we need to listen?" The discussion lasted 32 minutes altogether.

The network shows a relatively balanced structure between the coded competences. The network structure indicates that students were not only demonstrating these competences in isolation but were connecting them when speaking. The network is weighted slightly towards the lower left quadrant indicating that the competences of interacting with others and critical thinking were demonstrated more frequently during the P4C discussion.

Nodes which are positioned closer together are more often linked in the data as compared to those which are less linked. A triangle of thicker edges at the left side of the space connecting the competence nodes of character, respect for others , critical thinking and interacting with others illustrates that students predominantly demonstrated these competences and drew upon these competences together. This is further demonstrated by the relatively larger size of these nodes as compared to, for example, creativity which shows a smaller node in grey.

Displaying competences in the network model allows for interpretation of the way in which connections between the competences are being made, in addition to which competences are being demonstrated relatively more often. However, while studying the competences and connections between these competences in a single network can be illustrative in itself, notable characteristics become most apparent when networks are compared.

B/ Competence assessment between groups within a single class: Year two student vs teacher networks

Figure 3 and 4 depict network models constructed from the same P4C session dialogue data shown in Table 2. Models were produced to show the student's competence network (green) and the teacher's competence network (purple).

A subtracted network graph serves to highlight the differences between the students' competence network and the teacher's network (Figure 4). While students made stronger links between self-management and how to learn, as reflected by the more pronounced green edge connecting these nodes in the upper right quadrant, the teacher's network shows a very thick edge highlighted in purple between the competences caring thinking and provocative/debatable questioning. This suggests that the teacher was likely structuring the dialogue through a series of questions while also modeling caring thinking. Within the teacher network, there is also a clustering of utterances in the lower right quadrant surrounding these competences. For this reason, the teacher network mean (shown as a purple square) is positioned in the same lower right quadrant and the network is weighted towards this side.

In contrast, students had a larger spread of utterances linking competences, as demonstrated by the distribution of green dots across the projected space. There are however two notable triangles with a common vertex of critical thinking, linking respect for others and character in the upper left quadrant and self-managementand how to learn in the upper right quadrant. The student network is weighted towards the upper half of the space, with the mean shown as the green square found higher than that of the teacher network mean. Visually, it can be noted that there is a meaningful difference between the means of the student network as compared to that of the teacher network. Although the ENA tool is capable of computing inferential statistics related to the data, the current research design does not meet the fundamental assumptions underlying these statistics and therefore reporting them would be misleading and incorrect (White & Gorard, 2017).

C/ Competence assessment within a single class over time

Data from one year three class between September 2019 and March 2020 was used in order to illustrate how competences shown during P4C sessions could be assessed within a single class over time.

Table 3: Class 3A P4C sessions overview: September and March

Table 3: Class 3A (22 students, 12 boys, 10 girls, 1 male teacher)
P4C sessions overview: September and March

Due to the convenience sampling design of this study and the fact that by design, children create the questions within a P4C class, it was not possible for the class to discuss the same question for both lessons. As shown in Table 3, the questions are not identical. However, the competences communication, reflection, critical thinking, and collaboration can be compared between September and March as these competences were found to be most common to all P4C sessions regardless of the question or focus.

Figure 5 shows 3A's P4C session network from September in blue and March in red. The connections between the competence nodes reflection and critical thinking in both networks indicate a strong demonstration and co-occurrence of these competences in both P4C sessions. However, there was not found to be a significant difference between the two sessions as shown by the absence of an edge connecting these competence nodes in the subtracted network graph (figure 6). The means shown by the blue and red boxes are positioned close to one another signifying that the data did not suggest a large difference between the networks. Nonetheless, the subtracted network graph shows a greater co-occurrence between the competences reflection and communication in September in contrast to a greater co-occurrence between critical thinking and collaboration in March. Additionally, the subtracted network shows that overall there was a greater co-occurrence of reflection and collaboration, and communication and critical thinking in September. This may suggest that the dialogue in September was broader in that it was touching upon several competences simultaneously in comparison with the dialogue from March where the results suggest students were focused entirely on collaborating to think critically.

Although this example served to show the potential in comparing a single group over time, it should be noted that the time period between the sessions was limited. The goal was observational: to observe the state of P4C sessions at different time periods without manipulating any other factors, yet this also meant that the class was responding to two different questions and this may explain the difference in competences identified and highlighted in the results. It does however show that researchers may wish to consider the use of ENA in more robust longitudinal studies in the future.

D/ Competence assessment between classes: Year Three P4C Sessions

Classroom P4C data was analysed from two grade three classes. Both classes follow the same structure for P4C sessions. P4C sessions take place every two weeks at 13h30, after lunch and mindfulness. The session begins with students sitting in a circle. A chapter from Lipman's novel Elfie (2003) is read aloud, first by the teacher and then by the students taking turns. After reading aloud, the students have a few minutes to discuss the chapter and formulate a question related to the story. Once the questions are reviewed, the class conducts a blind vote, wherein each student votes for the question they would like to discuss that session. The question with the most votes becomes the question for the dialogue.

Data from the classes were compared using subtracted network models. In doing so, the most notable characteristics of each group become more evident. Competences which were most common between the dialogues were selected for network construction to compare how these competences differed between the classes. Nonetheless, it should be noted that although both groups read the same extract from the novel, the questions produced are not identical. These differences are due to the fact that students focus on different aspects of the story and create questions they find most interesting and relevant. This is shown in table 4 in the next section.

E/ Year 3 P4C Sessions - October 2019

In October, both year three classes focused on chapter two of the novel Elfie by Lipman (2003). They both followed the structure detailed in the previous section. Table 4 shows how each class responded to the stimulus.

Table 4: Year 3 P4C Sessions - October

Table 4: Year 3 P4C Sessions - October

*3A and 3B are not references to particular classes within the school. The A and B solely serve to distinguish between the different year three classes within this study.

Figure 7 shows the network of 3A in red, the network in 3B in blue and the subtracted network graph below. The weighted network accounts for the difference in the number of utterances between the classes and therefore enables comparison.

The networks of 3A and 3B have a very similar structure of connections. The thick edge connecting the competence nodes reflection and critical thinking in the lower right quadrant of both 3A and 3B's networks illustrates that these competences were often drawn upon together and were demonstrated more frequently than the other competences over the course of 3A and 3Bs philosophical dialogues. Although the overall network structure of 3B is similar to that of 3A's, illustrating that there were common competences demonstrated in similar ways by both groups, there are some slight differences between the networks. For example, it can be seen that 3A's network has, on a whole, heavier weighted edges between the nodes. This group was consistently demonstrating and linking competences throughout the sequence of their P4C dialogue. In contrast, 3B has thinner edges. However, 3B has a thicker edge connecting the competence nodes of negotiation and reflection. This illustrates that students within class 3B were negotiating with one another over the course of their dialogue.

The differences between 3A and 3B networks become more apparent when studying the subtracted network model (figure 8). The fact that this model is predominantly showing edges in red reflects that 3A connected these competences more frequently during their P4C discussion than the class of 3B. However, the edge shown in blue in the subtracted network model, between negotiation and reflection provides further support that students in class 3B were negotiating with one another over the course of dialogue more than those in class 3A. The means of 3A and 3B (indicated by boxes) are situated in close proximity to each other. This confirms the similar structure of the networks and suggests that dialogues in 3A and 3B showed similar features.

Studying these models revealed a notable difference in the way in which class 3A collaborated as compared to class 3B. A subtracted network model (Figure 8) derived by the decomposition approachshowed that 3B made more connections between negotiation and reflection as compared to 3A, highlighting that this class more frequently demonstrated the microcompetence of negotiation when collaborating with each other.

VI/ Discussion

The process of ENA outlined how the ISG curriculum competences can be defined, identified and assessed through the P4C programme. This was achievable through the creation of a codebook, coding of dialogue, and finally network modelling and analysis. The use of ENA served as a tool to explain student competences within a single P4C discussion and also highlight how competences can be demonstrated through different philosophical inquiries. Lastly, it showed that ENA may provide a means to view competence development over time within the students' evolution within the P4C programme.

The ENA networks allow us to view how the community of inquiry uses philosophical dialogue in order to draw upon the constituent elements of competence together and to connect these competences over the course of their dialogue. The value of ENA is in its ability to visually show the connections between the competences rather than solely the competences themselves, though it also serves as a descriptive tool for visually displaying what is coded. The network model should therefore be considered alongside the context of the philosophical dialogue. Researchers should start with the coded dialogue, create and study the network model and then return to the original P4C session in order to deeply understand where the connections in the model come from. The online ENA web tool allows the user to click on the connections between competence nodes and review the metadata that resulted in the connection (Shaffer, 2016). In doing so, the network becomes both the tool and the visual for understanding assessment within the P4C context.

In sum, ENA may help to assess the curriculum competences displayed during P4C dialogue through thick description: the method of closely studying connections in data in order to further understand how and why learning happens (Shaffer, 2018). ENA's potential for providing a holistic view of how students connect the constituent elements of competence within philosophical interactions can provide insight into how they learn.

Lastly, it can be noted how the P4C programme shares common goals with the ISG primary school curriculum. For example, one of the seven macrocompetences that the ULP aims to develop is interacting with others (UNESCO-IBE and International School of Geneva, 2018). It defines this competence as collaborating in order to address complex problems. In studying the P4C dialogue, interacting with others was frequently demonstrated and coded accordingly.

The P4C programme also aims for children to practice critical thinking by giving reasons and providing evidence (Fisher, 2007). Studying P4C dialogue provides a window into how students are able to express their reasoning.

Ultimately, the P4C programme is a pillar of the curriculum within the International School of Geneva and provides valuable insight into the way students learn while also helping them to develop as thoughtful individuals.

VII/ Implications for future research

Future research may wish to consider longitudinal studies in order to evaluate competence development over time, as this could not be confidently explored in the current study due to time constraints. By carrying out longitudinal studies, researchers can more accurately examine how network models of competence may change over time during the P4C programme.

Furthermore, although the ENA approach adopted within this study provided one example of how P4C sessions could be studied, multiple methods could be explored in order to capture the complexity and multifaceted nature of philosophical discussions.

VIII/ A final thought

Assessment, like philosophy, is not an exact science. It is a process. Only through the purposeful process of "observing, interpreting and recording evidence of learning" (Marope, 2017, p. 26) can we truly attempt to comprehend how students express their inner thinking. This study contributes to this collaborative and iterative process and hopes to inspire others to do the same.

(1) Katarina Hayek is an international primary school teacher with experience teaching various curricula in Switzerland, Italy and the United States. With degrees from McGill University in Canada and Durham University in the United Kingdom, she views research as a fundamental part of teaching and learning. Her most recent research, conducted within the International School of Geneva, focuses upon the Philosophy for Children (P4C) programme and investigates dialogical assessment using Epistemic Network Analysis (ENA).