counting grade 1 air quality days – a new metric for evaluating Beijing’s air quality?

Earlier this month, the Economist featured a nice piece about the ineffectiveness of current efforts to improve China’s environmental quality, including some discussion of recent decreases in air quality. The authors cite the official Ministry of Environmental Protection announcement for overall air quality averages in cities across the country, but run their own numbers for Beijing specifically, arriving at this conclusion:

Residents of Beijing, who hoped the clear skies they enjoyed during the 2008 Olympic Games would persist, have also resumed their grumbling. Smog is back with a vengeance…

Official data show a diminishing share of grade 1 (“excellent”) air-quality days since the games.

The claim is supported with this chart:

201032asc031

This chart caught my attention because it’s the first time I’ve seen “percentage of Grade 1 air quality days” used as an indicator of air quality. Usually, data for Grade 1 and Grade 2 are presented together. The reason is because China’s ambient air quality standard calls for urban air quality to meet only the Grade 2 level. (“Blue Sky Days” are days which have Grade 1 (excellent) or Grade 2 (good) air quality by the Chinese standards.) Charting percentage of days meeting Grade 1 or Grade 2 yields a very different impression:

number grade 1 and 2 through july 2010

Using this metric of Grade 1 or 2, there is no apparent decrease in air quality from 2008 to 2009, the drop from 2009 to 2010 seems minimal, and the overall situation appears much less dire (the percentage of passing days floats above 70% as opposed to 10%).

Two questions come to mind here. First, is it misleading for the Economist to use this unconventional metric? Maybe. The article neither describes what Grade 1 means nor justifies why “percentage of Grade 1 days” is an appropriate / better indicator of air quality, although Grade 2 is discredited using an comparison to the BeijingAir Twitter feed:

The American embassy in Beijing, to the annoyance of local officials, issues frequent air-quality readings for its part of the city. These, based on the presence of fine particulates, mostly ranged from “moderate” to “unhealthy” in the 24 hours after midday on July 31st. But the government called that period grade 2, or “good”. Incredibly, to anyone familiar with China’s perennially grey urban landscapes, fully 91% of days in 113 big cities in the first half of this year were described as “blue sky”.

Second question: is it correct to talk about percentage of Grade 1 days? That’s tough to answer. The technical answer is no, it’s not, because any use of “number of days” or “percentage of days” meeting a single standard is statistically meaningless. (For a short and nerdy explanation of this, see this post). That having been said, though, I do believe that scrutinizing Beijing’s air quality by looking at Grade 1 – instead of Grades 1 and 2 – days provides a much more reasonable snapshot of the actual air quality in the city. For one thing, China’s air quality limits for Grade 1 are much closer to internationally-recognized standards. For another, one would assume that there is lower risk for data manipulation around the Grade 1/2 border than around the Grade 2/3 border.

I raised the idea of looking closer at Grade 1 earlier this year. Maybe it’s time for a more in depth analysis, but I’ll save that for another post.

Leave a Reply