Wednesday, April 30, 2014

The Continued (and Failed) Quest for Automated Writing Assessment


It continues to surprise me how many are willing to sacrifice effectiveness for efficiency in education. Pearson Education, a well-known textbook publisher and notorious SAT scoring administrator turned education "innovator", has recently teamed up with schools across the country to gather data for their new automated essay scorer as part of the new Common Core Curriculum. "The company is using these student essays to train its robo-grader to replace one of the two human readers grading the essay, although there are no published data on their effectiveness in correcting human readers," says Les Perelman, the now-retired director of Writing Across the Curriculum at MIT. 

And this isn't the first time folks have tried to lay claim to the quality of robo-graders. Mark Shermis and Ben Hamner came fairly close to convincing a lot of people of the validity and significance of their findings regarding the near-human accuracy of their robo-grader – until Perelman blew them out of the water with his critique of their methods and data analysis (Rivard). In his critique, Perelman states, "The study’s methodology used one variable for comparing human readers and a different variable for comparing machine scores, this difference artificially privileging the machines in half the datasets." In his Boston Globe article today, Perelman was quick to point out that these "Robo Graders" are pretty good at counting words and determining vocabulary level, but not at understanding the complexities of thinking and understanding present in an essay. I find it odd that school administrators are willing to pay Pearson for a system that does something any third-grade student can do in Microsoft Word without paying a fee – check word count and check readability by grade level.

Many in our society set out on a quest for automation before fully understanding the phenomena and processes to be automated. Is this because we are waxing nostalgic for the Industrial Revolution? Is it due to of our collective American goal to be the leader in innovation (which to some equates with automation and technologization of just about everything)? Or is it, perhaps more realistically speaking, the human need for fame and fortune for inventing the next great thing? 

"Teaching to the test" is already an issue in our country. Let's hope writing instruction doesn't become an issue of "teaching to the machine" (that slept through fifth-grade composition).

Works Cited

Perelman, Les. "Critique (Ver. 3.4) of Mark D. Shermis & Ben Hammer, “Contrasting State-of-the-Art Automated Scoring of Essays: Analysis”." Ver. 3.4, Mar. 13, 2013. Web. 30 Apr. 2014.

Perelman, Les. "Standardized-test Robo-graders Flunk." BostonGlobe.com. Boston Globe Media LLC, 30 Apr. 2014. Web. 30 Apr. 2014.

Rivard, Ry. "Professors at Odds on Machine-graded Essays." Inside Higher Ed, 15 Mar. 2013. Web. 30 Apr. 2014.

Thursday, April 10, 2014

Need to Write? Head Outside.

In "Training Your Brain for Creativity," writer and fitness specialist James Fell passes along some wisdom that we can finally take advantage of now that one of the worst winters on record is finally coming to an end. A personal observation that his creativity improves while exercising outdoors--he says the bulk of his book was written while either on his bike or running--prompted Fell to talk with Ruth Ann Atchley, chair of the psychology department at the University of Kansas. 

"We've known for a long time that writers get benefit from being active out in nature," she explained. "The environment makes a big difference." And while it's long been established that aerobic exercise improves cognition, it's the combination of activity and the outdoors that's key: "Reading email and checking your phone takes you off task and inhibits creativity for as long as five minutes each time." In contrast, the "natural environment has the ability to seduce and attract your attention system rather than demand it." 

Exercise, outdoors or not, can also enhance your career. Fell cites neurologist Dr. Miguel Alonso-Alonso, who explained that a "fitter person is going to have greater improvements in executive function." Moreover, there may be a correlation between physical and financial health, with Fell reporting economist Vasilios Kosteas's finding that people who exercised at least three times each week increased their earnings: "I found that for men the average was a 6 percent increase on weekly earnings. For women it was more on the order of a 9 to 10 percent increase in weekly earnings," said Kosteas. 

To read the full article and in-depth explanations of the research, click here



Wednesday, April 2, 2014

April Is National Poetry Month

Yesterday marked the beginning of National Poetry Month. The celebration's organizer, the Academy of American Poets, has hundreds of suggestions for ways to celebrate. Here are a few highlights:   
  • Listen to readings of the site's most popular poems. (Click here to listen to the site's most popular recording, Dylan Thomas's "Do not go gentle into that good night."  
  • Explore the rich history of poetry here in Illinois. (Distance students, click here to find information about your state.) On each state-specific site you'll find biographies and the work of local poets, lists of "poetry-friendly bookstores," event calendars, information about local literary organizations and publications, and poems about the state, its cities, and its people.