The various features that are contained on middle school websites and the degree to which these websites reflect the overall mission of middle schools was studied using a checklist that could be summarized using quantitative descriptors. Although the websites were studied in a quantitative fashion, there were certain aspects of the evaluation such as download time and facility of site navigation that were subjective in aspect. These defining features of a website could be interpreted differently by different people. As Gladwell (2005) explains in Blink: The Power of Thinking Without Thinking, one’s “adaptive unconscious” (in other words, their very first impression) can “fundamentally shape” how something is viewed (p. 13). In respect to a website, the first viewing of the color scheme or placement of images might have unduly influenced the researchers in terms of rating the site in its entirety. It is clear that there was a degree of subjectivity, however small, in terms of the manner in which the quantitative results were derived. In order to address this potential for bias, the researchers ensured inter-rater reliability through each evaluating five websites that the other had also reviewed. These results were within three points on a 100 point scale.
To carry out the study, the researchers randomly sampled 50 middle school websites and offered their evaluation, based on a pre-determined checklist (see Appendix A). The criteria for what should be included (and consequently excluded) from this checklist was determined by a fusion of literature on the goals and mission of middle schools, the purpose of school websites, and general web design features.
In order to find middle school websites to examine at random, a Google™ search on “middle schools” was performed. The initial search found over 6 million websites that contained the exact phrase. To select which schools to examine, we simply chose 50 websites in a directed fashion. Google™ presented the search results in groups of 10, and a point was made not to choose more than one school from any 10 on one display page. In order to maximize the diversity of our findings, the researchers made an effort to examine middle school websites from different states. Each website was scrutinized by comparing items on the checklist to the site, and through interpretation of the general look, feel, and functionality of the site as a whole. Each of the websites’ scores was determined by giving one point for each of the design issues, and two points for each of the structure, content, and general issues. The maximum number of points possible from the scale was 100. This is illustrated in Table 1. It should be noted that the scoring system was the basis of the research itself. The overall evaluation of middle school websites took place subsequent to the amalgamation of the results from each individual middle school website.
Developing the Checklist
Checklist issues related to design were derived from the Web Style Guide, 2nd edition (Lynch & Horton, 2004). Major areas of focus included were interface design, page design, layout issues, graphic design, site design, and typography issues. Factors related to basic interface design were creating a user-centered design and ease of navigation. Page design issues that were considered included page consistency; having a visual hierarchy; page dimensions and length; page layout; and cross-platform issues. Also important to take into account were site design issues, such as organization of information and general site structure.
Graphics and typography issues such as location of graphics and text; alignment and size; typefaces; and consistency were also considered in the development of the checklist criteria. The four major web design principles of contrast, repetition, alignment, and proximity (Williams & Tollett, 2000) were considered. With these elements taken into account, a list of web design-related checklist items was generated. The list was then re-examined for elements that were unsuitable or repetitive. The final list of 34 design items was used in the Middle School Webpage Checklist.
The eight structural issues included in the checklist were derived from the Web Style Guide, 2nd edition (Lynch & Horton, 2004). The major areas considered in the development of these criteria were primarily interface and multimedia design. Issues related to navigation, size of graphics, and organization of information were taken into account in the development of the structural checklist criteria. The list of potential checklist items was developed and re-examined by the researchers. Items deemed unsuitable or repetitive were removed.
The 20 content elements criteria were primarily generated from a review of the literature related to both the goals of school websites and the goals of middle schools. Primary goals of middle schools include fostering communication between all stakeholders in the educational process; encouraging an interdisciplinary curriculum; providing a caring environment for students; and to “mediate between academic and social concerns” of students (National Middle School Association, 1996, p. 10). The primary goals of school websites include providing a wide array of information to site visitors and acting as an interface between schools, parents, and communities (McKenzie, 1997). Examining the parallels between these goals, general content criteria, and both parental and student content criteria were developed. These criteria were then examined and inappropriate or repetitive elements were removed.
The general elements criteria were developed through a cursory examination of a number of university and K-12 school websites and through discussions with K-12 teachers and parents. Criteria in the “general elements” section were items that various participants cited as important, but did not align with any of the other checklist categories.
Table 1: Middle School Websites: Evaluation Criteria Scale