Earlier this month, the Economist featured a nice piece about the ineffectiveness of current efforts to improve China’s environmental quality, including some discussion of recent decreases in air quality. The authors cite the official Ministry of Environmental Protection announcement for overall air quality averages in cities across the country, but run their own numbers for Beijing specifically, arriving at this conclusion:

Residents of Beijing, who hoped the clear skies they enjoyed during the 2008 Olympic Games would persist, have also resumed their grumbling. Smog is back with a vengeance…

Official data show a diminishing share of grade 1 (“excellent”) air-quality days since the games.

The claim is supported with this chart:

This chart caught my attention because it’s the first time I’ve seen “percentage of Grade 1 air quality days” used as an indicator of air quality. Usually, data for Grade 1 and Grade 2 are presented together. The reason is because China’s ambient air quality standard calls for urban air quality to meet only the Grade 2 level. (”Blue Sky Days” are days which have Grade 1 (excellent) or Grade 2 (good) air quality by the Chinese standards.) Charting percentage of days meeting Grade 1 or Grade 2 yields a very different impression:

Using this metric of Grade 1 or 2, there is no apparent decrease in air quality from 2008 to 2009, the drop from 2009 to 2010 seems minimal, and the overall situation appears much less dire (the percentage of passing days floats above 70% as opposed to 10%).

Two questions come to mind here. First, is it misleading for the Economist to use this unconventional metric? Maybe. The article neither describes what Grade 1 means nor justifies why “percentage of Grade 1 days” is an appropriate / better indicator of air quality, although Grade 2 is discredited using an comparison to the BeijingAir Twitter feed:

The American embassy in Beijing, to the annoyance of local officials, issues frequent air-quality readings for its part of the city. These, based on the presence of fine particulates, mostly ranged from “moderate” to “unhealthy” in the 24 hours after midday on July 31st. But the government called that period grade 2, or “good”. Incredibly, to anyone familiar with China’s perennially grey urban landscapes, fully 91% of days in 113 big cities in the first half of this year were described as “blue sky”.

Second question: is it correct to talk about percentage of Grade 1 days? That’s tough to answer. The technical answer is no, it’s not, because any use of “number of days” or “percentage of days” meeting a single standard is statistically meaningless. (For a short and nerdy explanation of this, see this post). That having been said, though, I do believe that scrutinizing Beijing’s air quality by looking at Grade 1 – instead of Grades 1 and 2 – days provides a much more reasonable snapshot of the actual air quality in the city. For one thing, China’s air quality limits for Grade 1 are much closer to internationally-recognized standards. For another, one would assume that there is lower risk for data manipulation around the Grade 1/2 border than around the Grade 2/3 border.

I raised the idea of looking closer at Grade 1 earlier this year. Maybe it’s time for a more in depth analysis, but I’ll save that for another post.

Last week, multiple media outlets (including Xinhua, the New York Times, and the Wall Street Journal) reported on MEP’s admission of worsening air quality during the first half of 2010. Here, I will take a closer look at how this was reported, what exactly this means, and what is happening in Beijing specifically.

First of all, the original announcement from MEP is here (Chinese only). Both Xinhua* and the Wall Street Journal made major mistakes in their reporting of the announcement. Both reported that the average atmospheric particulate level rose to 0.091 mg/m^3 from 0.002 mg/m^3 last year. This is incorrect. Doesn’t anyone check original sources any more? The MEP report says “可吸入颗粒物浓度同比上升0.002毫克/立方米。” This means that the PM level is 0.002 mg/m^3 higher than last year’s, not that last year’s value was 0.002 mg/m^3. Someone pointed this out in the comments section of the Wall Street Journal article (which claims the increase is “humongous”), but it still hasn’t been corrected.

The Wall Street Journal also made the mistake of claiming that “the first half of 2010 had the worst air quality since 2005.” This is also not true. MEP’s announcement, which the WSJ story links to, says, “自2005年以来，环保重点城市空气质量优良天数比例首次出现下降，可吸入颗粒物浓度首次上升。” The correct interpretation here is that the first half of 2010 is the first period since 2005 that there has been a decrease in air quality from one year to the next. In other words, the trend of improving air quality has changed. At least Xinhua got this one right:

Economic recovery has partly caused the country’s air quality to fall in the first half of the year, the first such fall since 2005, figures from environmental authorities showed on Monday…”It was the first time for these cities to record a fall in the number of days with good air quality and a rise in the concentration of inhalable particles since 2005,” ministry spokesman Tao Detian said.

Unfortunately, the WSJ’s mistakes are already bouncing around the blogosphere.

Of course, despite the mistakes I’ve pointed out, the story is still quite significant. After years of official statistics showing improving air quality, MEP’s air quality data now shows a very slight increase in ambient particulate matter concentration. The data MEP gives are average pollution levels for 113 major cities, which is a little strange, since pollution levels vary widely across China. Although the averages are an interesting snapshot (and it is significant that MEP is reporting this bad news), these averages are not very meaningful; they say nothing about regional trends / changes or population exposure. This is an important area for more detailed research.

MEP’s announcement also gives averages in percentages of days meeting Class I and II air quality standards (so-called “Blue Sky Days“), but this metric is meaningless, as described previously on this blog.

What is more interesting is looking at data on individual cities. After all, most people spend most of their time in a single city. Plus, local and regional pollution control programs may vary from place to place. For Beijing specifically, I ran my own analysis of Beijing’s API data for the first half of the year using data I downloaded from MEP’s datacenter. After converting API to PM10 concentration using the methodology described here, I calculated the average PM10 concentration for the first half of 2010 in Beijing to be 124 ug/m^3. I also produced the following figure:

The figure shows that Beijing’s air quality, using average PM10 concentration as an indicator, has not shown improvement over the period 2008-2010. This trend is a continuation of the stagnant pollution levels I described earlier this year. Although MEP’s 113-city average (91 ug/m^3) is below China’s ambient air quality standard (100 ug/m^3), Beijing’s air quality remains well above China’s own standard, which is well, well above the WHO’s.

For those of you keeping track of Blue Sky Days, the Beijing EPB announced that there were 140 Blue Sky Days in the first half of the year, which, despite the increase in air pollution, somehow means Beijing is still on track to meet its Blue Sky Day goal. But this is a subject for another post.

Anyone who read this blog over the period 2008 to 2009 will know that I often repeated a two-part mantra: Beijing’s air quality is getting better, but we have a long way to go. I always felt that acknowledging the first part was critical to making progress working with China, and I was happy to see the New York Times article last October making this exact claim (headline: “Beijing’s Air is Cleaner, but Far from Clean;” my analysis here) It is frustrating and sad that this is no longer the case, even by China’s official data.

*To be fair to Xinhua, their mistake was only one word. Their report states, “The amount of inhalable particles, a major air pollution index, was also 0.091 milligrams per cubic meter in these cities, rising from 0.002 milligrams per cubic meter over the same period last year, the ministry reported.” The problem in this sentence is the one word “from,” which significantly changes the meaning (from incorrect to correct) if removed.

A China Daily report today mentions Beijing’s 2010 Blue Sky Day target as 266. This is the first time I’ve seen the target referenced; I’m not sure how/when they set or publicize them. Perhaps they are in Beijing’s Five Year Plan. Does anyone know?

This China Daily report also mentions Beijing’s 2009 Blue Sky Day total, 285, representing the first time I’ve seen it mentioned in English-language domestic media (although Xinhua covered the accomplishment in Chinese on 1/1.)

Recent targets and totals are shown below. Note that the 2010 target is below the number of Blue Sky Days achieved in both 2008 and 2009.

Happy New Year (a few hours early)!!! For my last post of the year, here is some overall analysis of Beijing’s air quality over the past 12 months.

First, the numbers. As usual, I’m working from air quality data downloaded from the Ministry of Environmental Protection’s datacenter. (Direct link to Beijing’s data is here.) There is a bug in the system that prevents the query function from working properly, but you can still scroll through the data and download it manually through copy and paste.

The datacenter reports 363 data points for 2009; 2/18 and 11/14 are mysteriously missing. Assuming those two days were Blue Sky Days yields a Blue Sky Day count of 285 and a calculated average PM10 concentration of 120 ug/m3.

For anyone who is new to this blog or who needs a refresher, a Blue Sky Day is a day with an Air Pollution Index (API) of 100 or below, which means that the air quality meets China’s National Grade II air quality standard – theoretically “excellent” or “good” air quality. China tallies annual number of Blue Sky Days as a consumer-friendly and easily-understood measure of air quality progress, although the metric is prone to gaming and scientifically meaningless.

PM10 means particulate matter of size 10 microns or smaller, also called “inhalable particles.” In 2009, PM10 was the reported dominant pollutant on 97% of days with an API above 50. (No dominant pollutant is reported on days with an API 50 or below.) I back-calculated PM10 from API using formulas and assumptions described here.

Trends of Blue Sky Days and PM10 concentration since 2000 are shown here:

The Good News: The good news isn’t really that good, but in the interest of balance here are some positive perspectives. First, Beijing’s count of 285 Blue Sky Days in 2009 is well above that of 2008 (274), and also well above the goal of 259. This will keep the trending up in official reporting and for boosting public morale; I imagine this fact will be spun as success / progress in the Chinese media in the days to come. I suppose another bit of good news is that Beijing has achieved what appears to be an improvement over 2008 without the boost of two months of extreme traffic restrictions and factory closures implemented last year during the Olympic period.

The Bad News: My calculated average PM10 concentration in Beijing for 2009 is 120.2 ug/m3, which less than a 1% reduction from 2008’s average. In other words, from the perspective of average particulate matter concentration, there was no improvement in air quality in Beijing from 2008 to 2009. This shows the danger of using “number of Blue Sky Days” as a metric for air quality evaluation – just because the number of Blue Sky Days increases doesn’t mean that air quality has improved.

Interestingly, as a thought experiment, let’s suppose that China’s air quality goal was that all cities should meet the Grade I air quality standard, meaning that a Blue Sky Day would have an API of 50 or below as opposed to an API of 100 or below. If we count these days (shown in the graph above in light blue), we actually see that 2009 was worse than 2008; 2008 had 62 Grade I days, while 2009 had only 47.

Most importantly, regardless of how you spin the data, Beijing’s 2009 level of inhalable particulate matter was still 20% higher than China’s own air quality standard and six times higher than the WHO’s recommended annual PM10 standard. What this means, of course, is that our work continues to be cut out for us moving into 2010 and beyond.

Related post:  Summary of Beijing’s 2008 air quality

Although a little late, I do want to highlight a rather unexpected comment from the Beijing EPB at a press conference in early July. As reported in the Time blog:

[The Beijing EPB] spent much of their press conference Friday responding to doubts about the veracity of their numbers. Last year an American environmental consultant pointed out that the official numbers showed a disproportionately high number of days that fell just within the official target for a “blue sky day.” Yu Jianhua, head of Beijing’s environmental monitoring center, said the local government used emergency measures such as closing down construction sites on days when it expected pollution would exceed targets. That led to the high number of days just under the cutoff, Yu said.

This is the first time I’ve seen the Beijing EPB both directly acknowledge that a statistical anomaly in the number of blue sky days exists and provide a concrete reason why. The statistical anomaly I’m referring to can be be seen in this figure, from a 2008 report by environmental consultant Steven Andrews. The graph appears to show data biasing right around the “blue sky day” cut-off point (in this case, a PM10 concentration of 150 ug/m3).

It has been interesting to watch the evolving responses from the EPB to Mr. Andrews’ discovery – from official data – of what appears to be blatant data massaging to achieve an artificial result. In February 2008, Beijing EPB spokesperson Du Shaozhong infamously responded to criticisms of data manipulation with, “this phenomenon does not exist. ” By July 2008, in a press conference before the Olympics, the Beijing EPB response had shifted to the bewildering  “some convenience maybe taken in very adverse situations to improve the air quality within 9 square kilometers so that the API can remain at or below 100.” (The WSJ reported on this response the following day.)

Now, however, we have a definitive claim from the Beijing EPB that the preponderance of API values just below the blue sky day cut-off point resulted from emergency measures taken on days which were predicted to be dangerously close to the limit.

As is so often the case in China, this reponse only makes me ask more questions. Questions like:

- Why didn’t the Beijing EPB admit this last year, as soon as Mr. Andrews’ report came out?
- How are such accurate predictions made? Can we have more details on the program, like which factories or construction sites were closed?
- Why do this at all? Is there really a critical human health benefit to a 99 API day as opposed to a 101 day? (Answer: not really, since where we need to be is below 20.)
- Why were these emergency shut-downs conducted from 2003-2007, but not in 2008?
- Was this program really conducted in dozens of cities around China?

I can keep asking questions of course, but I think it’s time now to invoke Occam’s Razor in support of the more obvious conclusion…

More info in related posts on this blog:
October 2008: problems with the blue sky day metric
March 2009:  looking for biasing in 2008 blue sky day data
June 2009: new report shows widespread air quality data manipulation

Final note: I am playing catch up on posting after falling behind the last few weeks with the site redesign and work distractions. Apologies in advance that some of the commentary will be on “old” (e.g., from July) news.