A researcher developed rubric was used in the study. Two technology experts within the instructional technology and teacher education fields assisted the researchers in establishing the content validity of the rubric. The rubric was developed to provide a method of assessing the quantity and quality of reflection displayed in each subject's writing sample. Not only was the number of reflective entries counted, but the researchers also used the rubric to determine the depth of reflection found within each entry. The rubric used a five point scoring system where a rating of four (4) indicated evidence of meta-reflection, ratings of three (3), two (2), or one (1) indicated decreasing levels of reflection, and a rating of zero (0) indicated no evidence of reflection (see Appendix A). Adding the scores together resulted in an overall reflective score for each subject. The highest possible reflective score was 24.
All written materials were carefully read and reread by the researchers (raters), and the depth of reflections was noted using the researcher developed assessment rubric. Both raters were former K -12 teachers with extensive technology integration experience. One previously taught elementary and middle school science, while the other taught high school social studies. Both had conducted research previously on the efficacy of teacher-created, teacher-centered blogs. Each rater scored the data set independently. After scoring the data, the two raters compared results and discussed their viewpoints to maximize agreement in scoring.
Interrater agreement. The Pearson product moment correlation was used to establish interrater agreement for the two raters. The computed Pearson correlation coefficient r for overall scores was 0.815, indicating a positive relationship between the scores reported by the two raters. This result was deemed sufficient for the purposes of this study.
Do Blogs Support Middle School Teachers' Reflective Practice?
Data collected totaled 72 entries from 12 bloggers. Two raters analyzed each of the 72 entries, resulting in 144 individual scores. The total number of reflective entries found by rater one was 63 (mean = 2.14, SD = 1.26). The number reported by rater two was 56 (mean = 1.82, SD = 1.35) (see Table 2). Since the results were considered significantly and positively related, choosing either rater was deemed appropriate, but because rater two was a more conservative scorer, data analysis for this study was based on the results reported by rater two.
A majority (78%, n=56) of individual entries demonstrated some level of reflective writing. Fifteen (21%) of these entries scored 1 on the 4-point rubric while 17 entries (24%) scored 2. Fourteen entries (19%) scored 3 and 10 (14%) scored 4 (indicating meta-reflection) on the 4-point rubric. However, 22% (16) of the entries contained no evidence of reflection (see Table 3). Appendix B contains examples from the research sample that fit each category on the rubric.
Depth of Reflection
As stated previously, the highest possible reflective score (raw score) was 24. Individual scores for reflection ranged from a low of 2 to a high of 16. Mean (average) reflective scores for individual bloggers suggest that a low level of reflection occurred within the entries. While a single blogger's average was below 1 on the scale, five averaged between 1 and 1.99 and five others ranged between 2 and 2.9. None of the bloggers averaged higher than 2.66. The group mean of 1.82 (SD = 1.35) also suggests a low level of reflective practice (see Table 4).
The purpose of this research was to determine the usefulness of teacher-created, teacher-centered blogs at the middle school level. In particular, we sought to determine whether blogs supported reflective practice among middle school educators. Results indicated that a majority of entries examined demonstrated some level of reflective writing. Evidence collected from the analysis indicated that all participants engaged in some level of reflective writing.
Another research purpose involved determining the depth of reflection occurring within the entries. Results revealed that the depth of reflection varied markedly between and within individual blogs. Of the 56 entries demonstrating evidence of reflective practice, a majority (57%) did not support higher levels of reflective practice. Only 24 of the 56 entries (43%) scored 3 or higher on the rubric (see Table 3). This finding may be a result of the small number of entries examined. Further research containing a larger number of entries would be recommended.
Findings suggest that blogs have the potential to be effective reflective devices when used by middle school teachers. The majority of blogs examined in this study function as a place where teachers can write about and reflect on events occurring in their classrooms. However, only 10 out of 72 (13.8 %) entries demonstrate what Rodger (2002) referred to as a disposition for critically questioning what is going on in the classrooms (see Table 3). As Rodger argues, this particular disposition is critical if teachers are to successfully integrate theory into practice. It remains unclear whether blogs can promote higher levels of reflective practice among middle school teachers. Further research is needed to address this issue.
While blogs can be used to support written reflection, data reported in this study do not demonstrate that blogs are used exclusively for this purpose. Reflective practice appears to be only one of several reasons why middle school teachers blog. This finding is supported by previous research conducted by the authors (Ray & Hocutt, 2006). That blogs are used for a variety of purposes may also account for the low reflective scores reported here.
The results of this study give insight into the efficacy of blogs when used by middle school educators. These results demonstrate the potential usefulness of blogs in promoting reflective practice with practicing teachers. However, results do not demonstrate that blogs are being utilized effectively for reflective purposes. This conclusion suggests multiple avenues for future research. For example, research examining exemplary middle school teacher bloggers who do engage in frequent and deep reflection would be useful. Understanding the process in which these bloggers employ and why they use their blogs to support reflective practice would be illuminating. Research to support Bolton 's (1999) contention that reflective teaching practitioners are leaders and trendsetters would be beneficial as well.