As the NewsTrust Baltimore pilot winds down at the end of July, we are publishing a series of reports about our local news experiment. For the past six months, we have provided a free online service to help local residents find good journalism about their city -- and become more discerning news consumers in the process.
In this report, we’ll take a look at the site’s educational and community activities and make recommendations for future projects.
Our educational partners and community
To build community participation in NewsTrust Baltimore, we reached out to educational institutions and local nonprofits before launching our site. These partners agreed to use the site to engage their students to look critically at local news media, develop the skills to distinguish fact from fiction, and inspire their own writing.
During the pilot, we partnered with the Baltimore Algebra Project, Baltimore Civitas School, Baltimore Freedom Academy, Baltimore Urban Debate League, Loyola University Maryland, Morgan State University, Towson University, University of Maryland - College Park, and Wide Angle Youth Media. In total, we trained 26 teachers and facilitators from these organizations and conducted media literacy presentations for more than 200 students.
Student reviewers represented 64 percent of total reviewers in the first three months of our pilot, as shown in our pilot stats report. Most, if not all, students were affiliated with our partners. During our pilot, 179 student reviewers posted 1,413 story reviews as of July 10, 2011, making significant contributions to our site.
Building participation online through offline events
To establish NewsTrust Baltimore as a locally focused, community-driven site, NewsTrust conducted significant outreach at schools and in local venues. Community manager Gin Ferrara, local editor Mary Hartney, and writer/researcher Andrew Hazlett, along with NewsTrust founder Fabrice Florin and managing editor Jon Mitchell, met with community leaders before and during the pilot. Introductory presentations were made to community leaders, educators and journalists from November 2010 through January 2011.
On Feb. 2, two days after launching the site, NewsTrust Baltimore hosted a launch reception for all our partners at the Enoch Pratt Free Library, giving everyone the opportunity to meet face to face and to share their goals for the pilot. This event, along with our preliminary meetings, helped us connect with the community and encourage participation in the project, while also building new relationships among the partners. Another community event, a meetup, was held March 15 at Teavolve restaurant to connect with partners and members and get feedback on their initial experiences.
We also held a handful of awards events toward the end of the pilot, which we'll discuss later in this post.
In February, once the site was live and we were curating online events, we worked more closely with educational partners to train staff and students to use the site, traveling to their schools and offices to present to their communities. These presentations typically lasted between 45 minutes and 2 hours and included a group review project, in which students used NewsTrust Baltimore to review a news story, then discuss it, with moderation by NewsTrust staff. Most presentations concluded with a group discussion, when participants asked additional questions and shared their perspectives on local journalism. In total, we made 16 presentations to the staff and students of our educational and community partner organizations.
Our presentations and training sessions made it easier for people to start using the site. Students quickly became comfortable logging in, rating the news and writing story reviews. Having access to the NewsTrust Baltimore community manager and local editor in the classroom helped them overcome any technical challenges, and it allowed us to learn more about their own needs for the site. Modifications to group-page and profile-page templates were often made as a result of these classroom visits.
Students who were trained and participated in a group review generally continued to use the site after our visit, while most of the partners who had not chosen to have an in-person presentation used the site less. We had a far higher degree of participation from college students than high school students: 57 percent of our overall reviewers were college students, and 7 percent were high-school students.
Creativity in partner projects
Students were invited to review six different stories, including two test stories that were used to determine their news literacy before and after the program. Once students completed all six reviews, they could become Certified Student Reviewers if they passed our news literacy test. Certified reviewers were recognized as such both on their profiles and at an awards event at the end of their school year.
Beyond these recommendations, our educational partners integrated NewsTrust Baltimore into their curricula and schedule in various creative ways.
Towson University professor Stacy Spaulding used the site to help her students improve their own journalism in her “Writing for New Media” course. Students picked their own "beat," or topic area, to follow, and they reviewed a series of stories from various news sources on that theme. They then created blogs and news websites, where they published their own content. At the end of the term, our local editor was invited back to the class to be a judge of the final projects.
Wide Angle Youth Media, a nonprofit media education organization, took another approach, hosting a news hunt to find and review local journalism about youth. Wide Angle did this in concert with its annual youth media festival, providing its audience of youth, families and educators with an opportunity to extend the conversation about youth issues online and examine how local journalism report on youth. Wide Angle staffer Stephanie Dickard curated the youth page that week and found that NewsTrust Baltimore, "allows users to really look at news from a variety of perspectives and investigate what may be missing in an article."
At Loyola University Maryland, professor Stephanie Flores-Koulish integrated NewsTrust Baltimore into her "Media Literacy Education" course. Our pilot project was just one facet of her overall curriculum, which is a requirement for all graduate students in Loyola's literacy education program. Students reviewed three stories in preparation for a trip to the Newseum museum in Washington, D.C. Rashawna Sydnor wrote of the experience that "the critical analysis of the articles made me take a closer look at the framing and evidence presented, or the lack thereof, and I can clearly see how this could be a launch pad for middle to high school students to experience the same."
This summer, Baltimore Freedom Academy is using NewsTrust Baltimore to spark discussions with youth about freedom of speech, civil liberties and the law. Students in the ninth-grade seminar will review stories in these topic areas and participate on the Students and the Law group page.
Some ideas that were not implemented but are recommended for future news initiatives like this one include launching a youth-run online newspaper, conducting research and developing arguments for student debates, connecting student journalists from different schools through group pages and events, and developing a simplified review tool for working with younger students.
Measuring news literacy skills
Helping people become more critical news consumers is one of NewsTrust's key goals, and we worked to develop a prototype for assessing the news literacy of our students and the educational impact of this pilot. To that end, we identified two flawed news stories as our "pre" and "post" tests. Both stories were selected by NewsTrust editors as examples of bad journalism, with clear flaws that could be identified using our review tools. We asked our educational partners to have their students review the pre-test story at the beginning of the pilot, and we distributed the post-test story at the end, in conjunction with our community survey. We then compared student ratings to editor ratings: Student reviews that were within 1 rating point above or below our average editor ratings passed, and reviews that were outside that range failed our news-literacy test.
Our pre-test story, "Racial tension simmers on Martha's Vineyard as Barak Obama arrives" was a deeply flawed story that had been tested extensively on our national site and used in a range educational settings in the past (our editors gave it a 1.86 overall rating). We used two post-test stories, as some classes had already reviewed one: "Opting out - Kweisi Mfume not running in mayoral race, councilman says" and "News Flash: 14 year old Baltimore boy facing charges for beating grandmother in head with hammer" were both local news stories selected by NewsTrust staff during the pilot. In both cases, our editors gave "2" or lower ratings to at least three rating categories (e.g.: fairness, sourcing, depth), with an average editor rating of 2.15 for "Opting out" and 2.1 for "News Flash".
Of the students who completed the pre-test story review, 88 percent passed, or reviewed the story within the acceptable range. For the post-test story, 79 percent passed. This was a useful exercise, but we believe this prototype needs to be developed further in order to effectively show the skill development of students, for several reasons.
The pre-test story was often reviewed as part of our first group presentation, when our staff coached the students with helpful hints, and in many cases students had the opportunity to edit their reviews after the discussion of the piece. It is possible that some who gave the story high marks on the first pass changed their rating on reflection, or even continued to edit their review during the group discussion. In addition, the post-test stories were reviewed online and, in most cases, without group discussion of the articles, so we don’t have a sense of the environment the students were in or the attention they gave their reviews. It should also be noted that the post-test stories were not as deeply flawed as the pre-test story, which made it harder for students to discern their flaws. Finally, both the “pre” and “post” stories were public on the NewsTrust Baltimore site, and reviewers could read others’ ratings and reviews, which could influence their own thinking.
These factors contribute to our conclusion that the current version of our news-literacy test needs to be developed further before we can offer it as a standard tool for assessing news literacy.
That said, we can observe from these first prototype results that the majority of our student reviewers were able to review stories with a degree of critical thinking by the completion of the pilot.
Recognizing student and community achievement
It was our intention from launch to recognize the contributions of both our students and the NewsTrust Baltimore community at large. We developed awards for our college students (there was not enough participation by high-school students for awards by the school year's end), which we promoted to teachers and highlighted through our daily email newsletters and our blog.
Our education awards recognized thoughtful critiques, community contributions, and the trustworthiness of our reviewers, in these superlative award categories:
- News Hound Award: For contributing the most reviews to the site.
- Student Researcher Award: For posting the most stories from diverse sources to the site.
- Most Trusted Student Reviewer: For having the most reviews that were rated highly by NewsTrust Baltimore editors and the general community.
These awards were given to students who had already become Certified Student Reviewers (as described above), with only one award per category per school. Fifty-nine students from Towson University and one student from Loyola University were certified, and three students at Towson received one of the superlative awards.
We held an awards ceremony at Towson University on May 9, 2011, to recognize all the student achievers. It was an opportunity to connect once again with the community and to provide a fitting conclusion to their participation. At this time we also recognized the professors, whose commitment to the project encouraged so much student participation.
For other NewsTrust Baltimore members, a community awards event was held on June 21, 2011, at the Metro Gallery in Baltimore. Several members were recognized as "Top Reviewers," and one received an award as "Community NewsHound." These awards were given for either the highest number of reviews or posts, or for high ratings received by their peers for the quality of their reviews. We also gave awards to all our partners, guest hosts, and members who went above and beyond to make the pilot a success. This event was another opportunity for community members to network with each other and to build relationships outside of the online framework.
Considerations for future projects
Overall, we saw a high degree of college student participation throughout the academic semester. Once the school year ended, however, few students were able to maintain engagement. This tells us that educational activities should be scheduled with the school calendar in mind and that other community projects, such as Truthsquad, a fact-checking service, should be considered to maintain community participation in the summer months.
While many efforts were made to engage high-school partners, participation from this group remained comparatively low. Our outreach efforts may have been more successful had we begun the project in August, rather than January, so we could plan the partnerships and train the teachers before the start of the school year. Several presentations had to be postponed from January to later in February due to snow days, and by the time we worked with many teachers, they were shifting focus to prepare for the mandatory state exams.
High-school participation was also limited for many other reasons, as reported by teaching and program staff. Regular computer access was a challenge for some partners and potential partners. Some teachers felt that student technology literacy was low and that the process of teaching students to use the site could be challenging. Other teachers had difficulty integrating the pilot into their curriculum in the middle of the school year and might have had better success adding NewsTrust in the fall semester. Some students had poor reading and comprehension skills, making it difficult to evaluate the quality of the stories they were asked to review. Some partners expressed concerns about the relevance of the news sources to students; others were concerned about privacy controls on the site.
Susan Malone of Wide Angle Youth Media saw promise in using NewsTrust Baltimore with high-school youth, including "a collective effort to use the reviewing tools to evaluate media literacy skills of young people, but to do so youth need a more bite-sized way of digesting news media, with questions and fun facts to guide them in the process."
Some attempts to address these concerns included paper review forms for classes without computer access; limiting our high school workshops to explorations of facts and fairness, with new lesson plans; reducing the review format from “Full Rating” to “Quick Rating”; and identifying stories that have relevance to youth, in a variety of media types.
Other recommendations for future educational partnerships could be synchronizing pilot activities more closely with academic calendars; announcing awards criteria at the beginning of the pilot and offering prizes; developing targeted curricula for high-school and middle-school students; creating student log-ins with more privacy controls; and increasing the number of multimedia stories, along with special review criteria for multimedia journalism.
Conclusion: Educational structure and real-life connections support our online community
From our pilot, we learned that partnering with educational organizations enriches online community. It is clear that integrating NewsTrust Baltimore into the school curriculum boosts student participation and overall literacy, and we are grateful for the leadership of professors JoAnne Broadwater, Stephanie Flores-Koulish, Thom Lieb, Allissa Richardson, Stacy Spaulding and Ronald Yaros in promoting and sustaining student engagement.
Our student reviewers were quite self-motivated and seemed to enjoy this opportunity to learn the news of their community, while analyzing the strengths and weaknesses of local journalism. We also found that regular participation by students supported and encouraged thoughtful reviews by our general membership and helped to establish a civil online community.
By hosting live events and awards ceremonies, we were able to strengthen online connections and bring people together in the real world, not just on our site. This combination of education, outreach and recognition helped anchor NewsTrust Baltimore in the local community and made this service more useful for building skills, networks and access to information in Baltimore.
Photo credits: Fabrice Florin