Reflective Entry 8: Evaluate the outcomes of a digital and collaborative innovation in your practice from an educational research perspective.

Tiven, Fuchs, Bazari, and MacQuarrie (2018), state that there are three evaluation types important to understand what a program is accomplishing and how outcomes can be enhanced and strengthened: 

• needs assessments

• process evaluations

• outcome evaluations

I have considered this evaluation framework against the research I have undertaken and will make reference to the evaluation types throughout. 

Needs assessments

This 33-week journey has led me to the point of identifying and implementing a digital and collaborative innovation in my practice. This ultimately led me to explore the question:

If I use collaborative digital tools, how will it affect critical thinking (knowledge construction) and collaboration in my Year 1/2 priority learners? Key outcomes for me to evaluate from an educational research perspective concern whether there have been improvements in critical thinking (knowledge construction) and collaboration in my Year 1/2 priority learners as a result of them using collaborative digital tools.

Process evaluations

During the implementation of this innovation, there were a number of challenges that resulted in changes needed to be made to the implementation plan and for the innovation to pivot slightly, whilst still maintaining the integrity and working towards the intended outcomes. 

The student community within the classroom is made up of 14 students - 7 girls and 7 boys. I have identified 4 priority learners that I proposed to be the students that will most benefit from this innovation. I had planned for the implementation of my innovation to be staggered. This was based on my research into the Diffusion of Innovations Theory (Rodgers, 1962) and how innovations are communicated over time amongst participants in a social system. With knowledge of this I had identified 2 students within the classroom as early adopters to first trial the innovation with. It was then intended for these students to support other students by modelling their learning and collaboration on iPads and Explain Everything for the benefit of most of the remaining students - the early majority. Moore (1999), would define this stage of the innovation as “Crossing the Chasm” whereby the innovation could potentially fail due to its inability to reach the early majority. In reality, the innovation was unable to be implemented in a staggered way as a result of the classroom contact time that I had with the class, only working part-time. Instead, all students were exposed to the innovation at once and there was a surprisingly large number of students immediately feeling comfortable with this new way of learning. The students modelled a generally good level of the appropriate use of iPads and CyberSmart behaviours.

Outcome evaluations

The Collaboration and Knowledge Construction (Critical Thinking) 21st Century Learning Design Rubrics (ITL Research, 2012) were used three times throughout the implementation plan to measure changes in students against the rubric criteria. The data is set out on the two tables below:

Collaboration Rubric Score	Baseline	Midpoint	Summative
1
2	12
3	2	12	1
4		2	13
5

Knowledge Construction Rubric Score	Baseline	Midpoint	Summative
1
2	14
3		14	1
4			13
5

The first time the 21st Century Learning Design Rubrics (ITL Research, 2012) were implemented the findings were low scores for both collaboration and knowledge construction. The tasks the students were collaborating on required them to work together but the majority did not have shared responsibility. The students were also constructing knowledge, but that was not the main purpose of the tasks. 

The second time the 21st Century Learning Design Rubrics (ITL Research, 2012) were implemented the findings resulted in a shift for all students moving up a score in both collaboration and knowledge construction. Students had shared responsibility for the task but a majority were not required to make substantive decisions together during collaboration, however 2 students were. For knowledge construction all learners scored a 3 which meant the learning activity’s main requirement was knowledge construction but the learning activity did not require students to apply their knowledge in a new context. These findings informed changes to my implementation plan to see if I could elevate collaboration and knowledge construction further during the remainder of the innovation. 

The third and final time the 21st Century Learning Design Rubrics (ITL Research, 2012) were implemented, at the conclusion of the innovation, there had been significant shifts in the collaboration and knowledge construction rubric scores when compared against the start of the innovation. Most students, including all of my target students, were able to collaborate by having shared responsibility, and by making substantive decisions together about the content, process, or product of their work. In regards to knowledge construction, a majority of the students, including all of my identified target students reached a rubric score of 4. This means that the learning activity’s main requirement that they were engaged with was knowledge construction and it required students to apply their knowledge in a new context. Falloon (2015), suggests the mobility of an iPad supports effective collaboration by allowing students to work together in different spaces and places. However, simply having access to and using iPads didn't guarantee collaboration and critical thinking amongst students. The use of iPads and Explain Everything had to be embedded in learning that was purposely designed to develop collaboration and critical thinking skills (Falloon, 2015) and I believe the positive outcomes achieved in this innovation are as a result of that.

Student and Whānau Voice

A GoogleForm was administered to gather student and whānau voice in relation to collaboration and knowledge construction. It was clear from the initial form conducted part way through the innovation that students thought they were thinking critically and collaborating at a high level. This made me realise the need to further unpact these concepts with my learners and roleplay higher levels of collaboration and critical thinking. This change supported the findings of the final GoogleForm which indicated that as well as developing critical thinking and collaboration skills learners had further developed the Key Competencies (Ministry of Education, 2007) of ‘relating to others’ and ‘thinking’. 

As evidenced in the GoogleForm whānau have a much greater understanding of what critical thinking (knowledge construction) is and how they can support that both at home and school. This innovation changed the viewpoint of our whānau and further engaged them as an important learning stakeholder.

Future Focus 

Whilst there are many outcomes “specific changes in attitudes, behaviors, knowledge, skills, status” (W.K. Kellogg Foundation, 1998, p.8.) which changed as a result of my innovation there is still room to further develop the collaborative and knowledge constructing skills and dispositions in my class, as evidenced by qualitative and quantitative data gathered. In 2021, the next steps will be to provide for collaborative tasks that ensure learning is interdependent and the learning activities have learning goals in more than one subject. 

Conclusion 

Ultimately, despite its changes and the peculiarities of 2020, the innovation was a success as evidenced by whānau and student feedback and data. Therefore, it can be stated that the use of collaborative digital tools has positively affected critical thinking (knowledge construction) and collaboration amongst my Year 1/2 priority learners.

References

ITL Research. (2012). 21CLD Learning Activity Rubrics. Retrieved from https://www.academia.edu/34212053/21CLD_Learning_Activity_Rubrics

Ministry of Education. (2007). The New Zealand Curriculum. Wellington: Learning Media. Ministry of Education. 

Moore, G. (1999). Crossing the Chasm, Marketing and Selling High-Tech Products to Mainstream Customers. New York: HarperCollins.

Rogers, E. (1962). Diffusion of Innovations. Free Press of Glencoe.

Tiven, M. E., Fuchs, E., Bazari, A., & MacQuarrie, A. (2018). Evaluating global digital education: Student outcomes framework. New York, NY: Bloomberg Philanthropies and the Organisation for Economic Co-operation and Development. Retrieved from https://www.oecd.org/pisa/Evaluating-Global-Digital-Education-Student-Outcomes-Framework.pdf

W.K. Kellogg Foundation. (1998). Logic Model Development Guide. Retrieved from https://www.wkkf.org/resource-directory/resources/2004/01/logic-model-development-guide

Reflective Entry 7: Explain how your application of a digital and collaborative innovation relates to national educational policies/guidelines and contemporary educational theory and compare it to existing delivery models and needs.

My digital and collaborative innovation to improve students collaboration and critical thinking skills (knowledge construction) was guided and enhanced by several national educational policies/guidelines and informed by contemporary educational theory during its application. 

Policies and Guidelines

The policies/frameworks that impacted on and informed my digital and collaborative innovation included the following:

• e-Learning Planning Framework (eLPF)

• Digital Technologies in The New Zealand Curriculum

• National Education Goals

• Ka Hikitia – Ka Hāpaitia | The Māori Education Strategy

I first explored the e-Learning Planning Framework (eLPF) (Ministry of Education, n.d.) during the initial phases of my digital and collaborative innovation. This was partly to familiarise me with the tool and to also consider where my school and my practice was in relation to the eLPF.

The framework is designed to help schools and teachers reflect on and evaluate their e-learning capability based on five phases - pre-emerging, emerging, engaging, extending and empowering. These phases have “been aligned with a number of international frameworks that describe how technology is adopted and integrated into teaching and learning” (Ministry of Education. n.d). After reviewing the eLPF, I considered that the school was generally well placed and most practice would be engaging or extending in terms of the phases. The school benefits from effectively aligned processes and practices across the school and community. Therefore my innovation was able to build off this sound platform. I considered that my innovation aligned with the engaging phase as I endeavoured to use digital technologies appropriately to support higher-order (deep/critical), collaborative teaching and learning. Students were required to work collaboratively on iPads using the app Explain Everything to construct knowledge around the theme of energy. Multimodal resources were able to be utilised to enhance and build learners’ conceptual knowledge and critical thinking. This would have only been possible by using digital tools. In order to meet the extending phase, I would need to have established authentic collaboration with students to develop co-constructed learning throughout the innovation. Further work on this in the future could see this become a reality.

The New Zealand Technology Curriculum with the addition of Digital Technologies and Hangarau Matihiko learning (Ministry of Education, 2018) sets out the expectation that learners will be scaffolded to become competent users of digital tools, enabling them to be active members of society. The National Education Goals (NEG) 3 and 5 also talk about the importance of students developing a high level of knowledge, understanding and skills so that they can be successful in the modern world (Ministry of Education, 2019). My innovation sought to develop collaboration and critical thinking (knowledge construction) while simultaneously developing learners’ exposure and knowledge of different digital technologies. This innovation allowed learners to be exposed to different digital tools, resources and applications and to create their own digital outcomes. Explain Everything was used on iPads as a tool for collaborating, accessing and creating learning and thinking critically. 

Ka Hikitia (Ministry of Education, 2017) is a cross-agency strategy for the education sector to achieve system shifts to ensure that Māori are enjoying and achieving education success as Māori. This strategy highlights the benefits of establishing and maintaining strong and reciprocal relationships with Māori learners and their whānau. This approach benefits not only Māori learners but all learners. There are five outcome domains to support excellent outcomes for Māori learners and whānau. They are;

• Te Whānau: Education provision responds to learners within the context of their whānau.

• Te Tangata: Māori are free from racism, discrimination and stigma in education.

• Te Kanorautanga: Māori are diverse and need to be understood in the context of their diverse aspirations and lived experiences.

• Te Tuakiritanga: Identity, language and culture matter for Māori learners.

• Te Rangatiratanga: Māori exercise their authority and agency in education.

I believe these 5 domains were well catered for within my innovation. Specifically, this innovation provided for enhanced relationships with my learners’ whānau to develop and encouraged whānau to engage and participate as an equal part in the education of their ākonga. Grappling with this issue of whānau participation throughout the innovation resulted in me significantly adapting the approach taken towards engaging whānau. My aspiration was for them to be active participants from the onset of the innovation. This morphed into active supporters as the innovation continued. Continuing to implement practices that provide for learners to learn within the context of whānau will be imperative moving forward.

The contemporary education theories that impacted on and informed my digital and collaborative innovation included the following:

• Connectivist Learning Theory

• Social Constructivist Theory

Connectivist Learning Theory or connectivism is underpinned by the notion that knowledge is created through connections and learning occurs through the networks to which learners belong (Starkey, 2012). The idea that the teacher is the font of all knowledge is challenged and instead collaboration and connections between learners, learners and community, and learners and experts beyond the classroom. Within my innovation, learners worked collaboratively to create new learning. They were involved in shared learning experiences regarding the topic of energy, recording and discussing their ideas in groups, allowing students to construct knowledge and think critically about the topic. This aligns with connectivism, where learners can access and gather information from and share information with the rest of the world (Siemens, 2005) & (Downes, 2010). 

Social Constructivist Theory of constructivism views learners as “‘constructors’ of meaning, actively seeking to discover and learn” (Krause et al, 2010, p.198). Within social development, it can be linked to Lev Vygotsky, who believed that knowledge is co-constructed and that individuals learn from one another (McLeod, 2014). Vygotsky identified the zone of proximal development, which is the difference between the learner’s current level of understanding or skill and the level they can achieve with support (Krause et al, 2010). My innovation actively embraced constructivism and explicitly encouraged students to construct knowledge and demonstrate that by utilising digital technologies. Learners were also supported with extension and guidance when needed by either their peers or the teacher, ensuring learners were operating in their zone of proximal development.

Throughout my innovation, there were a variety of delivery models. A combination of blended, online and self-directed learning was used within each lesson. Students were encouraged to use iPads and Explain Everything to capture their ideas so they could use video and audio recording, images, photos and voice to construct knowledge. This allowed for blended learning as some students were using a variety of other tools to support their learning, such as creating a model using paper and then photographing it on the iPad to share on Explain Everything. All explicit teaching and learning was made available through the learning site online and was fully rewindable, meaning students could rewatch the video of the lesson as many times as they would like, promoting self-directed learning. This was generally well-received considering the students were year 1/2 learners. They appreciated the variety while maintaining an element of structure throughout the innovation.  

References

Downes, S. (2010). New technology supporting informal learning. Journal of Emerging Technologies in Web Intelligence, 2(1), 27-33.

Krause, K., Bochner, S., Duchesne, S., & McMaugh, A. (2010). Educational Psychology for Learning and Teaching (3rd ed.). South Melbourne, Victoria, Australia: Cengage Learning Australia.

Ministry of Education. (n.d.). E-learning Planning Framework. Retrieved from

http://elearning.tki.org.nz/Professional-learning/e-Learning-Planning-Framework#framework

Ministry of Education. (2017). Ka Hikitia. Retrieved from: https://www.education.govt.nz/our-work/overall-strategies-and-policies/ka-hikitia-accelerating-success-20132017/the-maori-education-strategy-ka-hikitia-accelerating-success-2013-2017/

Ministry of Education. (2018). Digital Technologies and Hangarau Matihiko learning. Retrieved from https://www.education.govt.nz/our-work/changes-in-education/digital-technologies-and-hangara  u-matihiko-learning/

Ministry of Education. (2019). National Education Goals. Retrieved from https://www.education.govt.nz/our-work/legislation/negs/

Siemens, G. (2005). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2(1), pp.3-10.

Starkey, L. (2012). Teaching and learning in the digital age. ProQuest Ebook Central. Retrieved from: https://ebookcentral.proquest.com