I’m a couple of weeks late on this (still playing catch up), but I thought this recent story was worth noting.

On May 2nd, China Daily reported that “Beijing has ‘cleanest month’ in 9 years,” writing, “the city experienced its best month of air quality since 2000 with 23 blue-sky days in April.”

The excellent blog Daily Dose of Air Pollution highlighted that this claim is dubious, noting at least three other months (August and September, 2008, and August 2006) in which Beijing had higher numbers of Blue Sky Days and lower average APIs than April 2009.

I think I’ve identified the source of confusion. The official Beijing EPB announcement (Chinese), titled 4月本市空气质量创2000年以来同期最好水平, states specifically that April 2009 was the best April since 2000, not the best month overall. It seems the China Daily (or the Beijing EPB spokesperson during the press conference) misrepresented the real announcement.

Two follow up points:

1) While acknowledging progress, we should also simultaneously not get too excited over the “clean” air. The Beijing EPB claims that the average PM concentration during this month was 120 ug/m3 (主要污染物可吸入颗粒物月均浓度为每立方米0.12 毫克), which is still well above China’s national air quality target (100 ug/m3) and six times higher than the WHO recommended guideline (20 ug/m3). (Comparison of international standards in this post.) Although it is critically important in China to note progress, we must not wrap ourselves so much in cheers of success that we become blinded to the significant challenges and work still ahead.

2) The China Daily article describes in more detail than I have ever seen how the economic slowdown may have contributed to improved air quality, writing:

Besides strict environmental protection measures, experts think the global economic slowdown might be playing a positive role in environmental protection.

Zhu Tong, an environment professor with Peking University, told China Daily on Friday that heavy industry has decreased production in many polluting factories, which benefits the air.

“Most companies in heavy industry are seeing fewer orders. The output of the Shougang Group this year so far equals the same period during the Olympics,” said Wang Dawei, head of the air quality control division of the Beijing municipal environmental protection bureau.

In the first season this year, the added value for ferrous metal and chemistry manufacturing in the capital was 3.36 billion yuan ($490 million) and 1.85 billion yuan, a year-on-year decrease of 18.1 percent and 17.9 percent respectively.

If the improved air quality is indeed due to the slowdown, then this means there is even less cause for celebration.

Yesterday, Xinhua reported that Beijing achieved above an 80% “blue sky” rate in the first half of this year:

BEIJING, March 31 (Xinhua) — Beijing saw 73 “blue sky days”, 81.1 percent of the total, in the first three months of 2009, Beijing authorities said here on Tuesday.

The city experienced six more blue sky days than in the first quarter of last year, and 24.3 days more than the average of the last decade, said an official of the Beijing Municipal Environmental Protection Bureau.

Experts at the bureau said active cold airflows had helped particulate matter to disperse.

Efforts to reduce pollution from heating systems had paid off with January having the most blue sky days since 2000, said the official.

For those who prefer original sources, here is the notice from the Beijing EPB (in Chinese).

Let’s take a closer look at the data and examine what this means in terms of air quality.

First of all, as a quick introduction for new readers, China defines “Blue Sky Days” as days for which the Air Pollution Index (API) is 100 or below. For a detailed description of exactly what the API is, see this post. Past Beijing API data may be queried from MEP’s datacenter, but unfortunately only in Chinese.

For this post, I queried the data for 1/1/09 to 3/31/09, and ran some quick analyses to see what interesting things I could find.

Part 1: Checking Beijing’s EPB’s Numbers

The Beijing EPB claims “截至3月31日，今年累计73个达标天，占监测天数的81.1%。其中一级7天，二级66天，三级15天，四级1天，五级1天。” The second sentence says that in this time period there were 7 days of Grade 1 (API 0-50) air, 66 days of Grade 2 (API 51-100) air, 15 days of Grade 3 (API 101-200) air, and 1 day each at Grade 4 (API 201-300) and Grade 5 (API 301+).

However, by my count, there were 8 days of Grade 1 (one more than reported) 64 days of Grade 2 (two less than reported), then 15, 1, and 1 days of Grades 3, 4, and 5, respectively (identical to that reported). It seems pretty basic to me that you would want your publicly reported data summary to match your public database, so I can’t imagine what’s going on here. This is especially true because, in this case, the data indicates Beijing did better than they claimed.

The data for one day, 2/19/09, is missing from the database. If we assume that 2/19 was a Blue Sky Day, though, then at least we do indeed have 73 Blue Sky Days for the quarter.

Part 2: Converting to Pollutant Concentrations

Because API is a unitless index, in order to evaluate air quality we have to convert back to pollutant concentrations. This is a bit tricky, because the API is only reported according to whichever pollutant had the highest daily concentration, meaning that we do not have daily raw data for every pollutant. Still, we can make a rough approximation by assuming that PM10 is the limiting pollutant on all days. (Of the 89 data points, 72 (81%) were reported with PM10 highest, 9 were reported with SO2 highest, while 8 had no pollutant data because no pollutants are listed for Grade 1 air quality days.)

In any case, given the above assumption, this graph shows calculated daily PM10 concentrations for this quarter:

These data yield a quarterly average PM10 concentration of 124 ug/m^3. This is very similar to what I calculated as last year’s annual average.

124 ug/m^3 is still well above China’s annual target (100 ug/m^3) and well well above the WHO’s ideal target for developed nations (20 ug/m^3).

Appendix

For those of you following along at home, here is an Excel formula for converting API to PM10 concentration (in this example, the API would be in cell E9):

=IF(E9<51,e9,if(e9<201,(e9-25)/0.5,if(e9<301,(e9+300)/1.429,if(e9<401,(e9+225)/1.25,e9+100))))

This is based on equations presented at the bottom of this post.

Sorry about the deluge of posts today. I’ve recently returned to China after some time away in the States and am playing catch up on everything I missed.

Here I’m posting two additional databases of Chinese APIs outside of MEP’s datacenter. Neither database has new data outside of the government-issued API data, however.

First, the Clean Air Initiative for Asian Cities has excellent China air quality data sets available for downloading. For example, Beijing APIs from 2000 to mid-2008 (along with tons of analysis) available on this page. Note that data for other cities may be downloaded at the bottom of the page. Note also the excellent API to pollutant concentration converter on the right.

Second, in late December, Imagethief posted “all the data you ever wanted on Beijing and Shanghai air pollution.” In the post, he links to a 5MB Excel file containing daily API data for Beijing and Shanghai from 2000 through mid-2007. The spreadsheet also contains a lot of analysis.

Some day I’ll post my own spreadsheets, but that will require me to clean them up (a lot) and also figure out a way to share files easily. I’m sure there’s a way, I just haven’t looked into it yet.

In any case, as you cut through the data from the above sources on your own, let me know what you discover!

The Asia Society has an excellent website that features daily photos of Beijing’s air, along with API info and a great video on the challenge of parallel economic development and environmental protection.

Happy New Year! I thought I would start off the year with a brief look back at Beijing’s air quality during 2008.

On December 31st, Xinhua reported that Beijing had achieved 274 “Blue Sky Days” in 2008. This was well in excess of the 2008 goal of 256, and even well above the 2009 goal of 259. But what does it mean in terms of air quality and human health? Let’s take a closer look at the data to find out.

First of all, according to my tally, Beijing actually only achieved 272 Blue Sky Days in 2008, with one data point (9/6) missing. Even if we assume the 9/6 sky was blue though, that only amounts to 273, not the reported 274. What’s going on here? FYI, I performed my tally by first downloading Beijing’s 2008 API data (available by querying MEP’s datacenter) then counting the number of days with API 100 or below. Am I doing something wrong here?

–
Update 1/15/09: In reviewing the data, I realized that MEP’s 2008 API database is missing two data points – 9/6 and 6/4. Assuming both of these days were Blue Sky Days yields 274. I missed this the first time around because I forgot that I should be looking for 366 total data points (leap year!) not 365.
–

In any case, as I have written about before, the “Blue Sky Day” metric is problematic for several reasons. Perhaps what bothers me most about it is that it tells us nothing about actual air quality; increasing annual numbers of Blue Sky Days does not necessarily mean better air quality(1). To evaluate air quality, we need numbers for daily / annual concentrations of air pollutants. Although the Beijing EPB publishes annual pollutant concentrations in the Beijing Environmental Annual Reports, the 2008 report won’t be available until this summer. So we need to improvise:

Starting from the database of 2008 API values, I converted back to daily PM10 concentrations using the formulas at the bottom of this post. I assume that the primary pollutant on all days is PM10(2). Averaging over the year I get:

2008 Average PM10 concentration for Beijing: 123 ug/m^3.

–
Update 10/29/09: Official statistics have been released, and show a 2008 annual average PM10 concentration of 122 ug/m^3.
–

The good news? This is a 17% improvement over last year. The bad news? The PM10 concentration is still over six times higher than the WHO annual target of 20 ug/m^3:

During the Olympics, Beijing saw a 50% reduction in air pollution as the city enjoyed its cleanest air in ten years. Clearly, the success of the anti-pollution campaigns was a driving force behind 2008’s relative improvement over years past. At the same time, though, we have a long way to go, and the considerable pollution of the city even in a “successful” year like 2008 should not be underestimated.

(1)Here, I’m not referring to data biasing. Rather, I’m simply considering the fact that Blue Sky Days are binary, as opposed to being a concentration value or gradual scale. Consider this extreme situation: if every day in one year had an API of 100, though the number of Blue Sky Days would be 365, the average annual PM10 concentration (indicating air quality) would actually be worse than it was in 2007 or 2008 in Beijing.

(2) This assumption is slightly problematic because for APIs below 50 the primary pollutant is not listed, although for APIs above 50 the primary pollutant is almost always PM10. To estimate the accuracy of this method, I used it on the 2006 and 2007 daily API databases and calculated an annual average PM10 concentration result for each year that deviated from the Beijing EPB’s reported values by well under 1%. Therefore, I think the assumption is pretty reasonable.

Earlier this month, the Beijing EPB announced that Beijing reached the 2008 target for total number of “blue sky days” one month early. A “blue sky day” is defined as one for which the API is below 100, indicating “excellent” (优) or “good” (良) air quality.

The stated goal of this blog is to “explore miscommunication between China and the West on issues of the environment and beyond.” As such, I want to highlight a major difference between how this story was reported within China and internationally.

Specifically, let’s look at the Xinhua English language report and the China Daily report (domestic) vs. the widely distributed Associated Press report (international). As expected, the domestic sources are purely positive. Xinhua’s lead is simple and dither-free:

Beijing fulfils “blue sky” day goal one month in advance
Beijing has met its 2008 target of 256 blue sky days as Sunday marked another day of good air quality.

China Daily’s lead is even more decisive:

Clear days’ target met before time
Determined efforts, special measures and good weather helped Beijing achieve its annual target of 256 blue-sky days yesterday, a full month before the end of the year.

Compare these to the AP’s lead:

Beijing claims early victory over air pollution
BEIJING (AP) — Beijing said Monday it has already reached its target number of 256 “blue-sky days” this year, with the help of ambitious environmental measures the city imposed to cut emissions for the Olympic Games.

Two words – “claims” and “said” – in the AP lead jump out at me as setting a very different tone from the Xinhua and China Daily stories. The implication being that the truth may be different from what the Beijing EPB proclaims. And, indeed, this suspicion over data integrity is substantiated in the last three paragraphs of the AP story, three paragraphs that are conspicuously absent from the domestic China reporting:

Steven Andrews, an independent environmental consultant based in Washington, said Beijing’s claims of improved air quality are not reliable because the city has moved monitoring stations to less-polluted areas and has varied the way it has measured pollutants since 1998.

“They’ve measured different things during that time period and it has a huge impact on the number of days that meet the national standard,” Andrews said in a telephone interview.

Such inconsistencies mean that the increase in the number of blue-sky days may be due to the change of monitoring locations, rather than a reduction in overall pollution levels, he said.

For me, the discrepancy between the international and domestic reporting is deeply frustrating on several levels.

Perhaps most importantly, I am frustrated that Beijing’s air quality data is (justifiably) not considered trustworthy by the international community, and that the Chinese media is forbidden (or just blind) to questioning it. As I have posted before, Steven Andrews’ excellent analysis of Beijing’s historical air quality data strongly suggests past data manipulation, in addition to his reasonable claim that one cannot directly compare the number of Blue Sky Days from different years due to monitoring station location changes. Until the Beijing government takes concrete steps to improve data transparency and independent confirmation, questions of data trustworthiness should and will remain.

On the other hand, though, I am frustrated because, due largely to the two-month shut down surrounding the Olympics, this is a year in which Beijing’s air quality genuinely has been better. In other words, this is a year in which Beijing did not need to, and may not have, engaged in data manipulation to meet targets. Any yet, at a time when Beijing arguably deserves credit for achieving its Olympic air quality goals (and for taking steps to maintain air quality in the post-Olympic period), there still seems to be a strong sense in the international community that any success Beijing achieved is either fake or was achieved by cheating. Virtually every conversation I have with a non-Chinese about Beijing’s air quality begins with, “well isn’t it true that Beijing’s air quality data is bogus anyway?”

In relation to the AP article, consider this statement:

Such inconsistencies mean that the increase in the number of blue-sky days may be due to the change of monitoring locations, rather than a reduction in overall pollution levels, he said.

While it may be true that the change in monitoring stations affected the number of days considered Blue Sky Days, this does not necessarily mean that there was not a reduction in overall pollution levels from 2007 to 2008.

My final point of frustration is really a technical issue with the number Blue Sky Days metric. Simply put, the number of Blue Sky Days metric is meaningless from a human health perspective, since what matters is average concentration of pollutants, not number of days below an arbitrary cut-off point. The value of the number of Blue Sky Days metric is not scientific, it’s social; it’s a way of packaging air quality information into a format thought to be easily comprehended by the public. I mention this only to say that what I am really curious about regarding 2008 is not number of Blue Sky Days, but rather average annual particulate concentration. It is only with that data (combined with whatever additional independent confirmations are available) that we will be able to make any real judgments about changing air quality.

Image: China Daily

Late last month, consultant Steven Q. Andrews published an excellent report with detailed analysis of the publicly reported daily API data for Beijing. His key findings, which were first published as an op-ed in the Wall Street Journal (and subsequently covered by the New York Times, Time, and others), are as follows (quoted from the report’s abstract):

Here I show that reported improvements in air quality [in Bejiing] for 2006–2007 over 2002 levels can be attributed to (a) a shift in reported daily PM10 concentrations from just above to just below the national standard, and (b) a shift of monitoring stations in 2006 to less polluted areas.

Many people, including Mr. Andrews, have been asking me my opinion on the findings of the report. Here, I will try to summarize what I consider to be the most impressive and surprising results, while also commenting on what the report’s results do – and don’t – show.

Summary: Statistical analysis of reported API frequency shows clear data biasing to reach Blue Sky Day targets, and, as such, I think effectively invalidates the use of annual number of Blue Sky Days as an air quality measure. This is an extremely impressive finding that I hope the Chinese government will respond to appropriately, both by reconsidering the use of the Blue Sky Day metric altogether and investigating how such bias was introduced and eliminating it in the future.

However, I caution against using this finding alone to make broader, sweeping assumptions about Beijing’s air quality changes over the last five or ten years. Specifically, Mr. Andrews’ report should not be used as proof that any and all recent improvements in air quality in Beijing have been simply the result of “gaming the numbers” as opposed to actual improvements. While Mr. Andrews’ results regarding numbers of Blue Sky Days are dramatic, his analysis of pollutant concentrations in recent years focuses on only one pollutant, PM10, and shows concerning, but not dramatic, discrepancies to the officially reported data. Additionally, when discussing PM10, it should be noted that controlling concentrations of particulate is well known to be one of the biggest air quality challenges faced by Beijing, and that trends of PM10 concentration should not necessarily be equated with trends of other pollutants or even trends of overall air quality.

Finally, while I think Mr. Andrews’ analysis normalizing air quality data across multiple years by accounting for the moving of monitoring stations is excellent, correct, and appropriate, I do not think it proves deliberate deceit about air quality (as the Blue Sky Day biasing does).

Discussion:

Blue Sky Day Data Biasing

First of all, to me, the most impressive graph in the report is this one, showing the dramatically higher frequency of reported PM10 concentration just below the Blue Sky Day cut-off than just above (Figure 2 from Mr. Andrews’ report):

Equally impressive is this statement from the report:

While 52% of the days with a city API between 96 and 105 (PM10 = 142–160 μg m−3) were reported as ‘Blue Sky’ days in 2001, 98% of the days in this range were ‘Blue Sky’ days in 2006, and 93% of days in the range were ‘Blue Sky’ days in 2007.

This seems to show, unequivocally, that there is bias in the reported data around the Blue Sky Day cut-off point. From this data, it appears that the use of the number of Blue Sky Days metric is not reliable as an indicator of Beijing’s air quality improvement. Therefore, I will stop using it as such and will edit a previous post on this blog that references it. I would hope that, in time, Beijing will recognize this clear bias and take steps towards identifying how it is introduced and preventing it in the future. At the same time, detailed investigations into potential biasing of other pollutant data should also be conducted (though it seems less likely that such biasing would have occurred, given the fact that the biasing appears to be related to meeting Blue Sky Day targets, for which PM10 is usually the limiting factor).

Moving of Monitoring Stations

The second issue Mr. Andrews raises is the moving of monitoring stations. While I find his analysis here to be fascinating and correct, I’m not convinced that his results prove that the moving of the monitoring stations was driven by the desire to lower artificially air pollution levels by measuring in less polluted areas. He mentions that the new monitoring regulations put into effect in 2006 included “new specifications…regarding the minimum distance from roadways that air pollution should be monitored.” I don’t know enough about international monitoring to know if perhaps these new standards were designed simply to bring China’s monitoring better in line with international standards? Whatever the case, Mr. Andrews’ point that different measuring systems were used is valid:

It has been widely reported that the number of ‘Blue Sky’ days in Beijing increased from 100 in 1998 to 246 in 2007, but these reported trends encompass a period during which air quality was evaluated in three different ways: (1) 1998– 1999, based on the 1996 Chinese national ambient air quality standards (2) 2000–2005, based on the 2000 revisions of the Chinese national ambient air quality standards and using the 1984–2005 monitoring station locations (3) 2006–2007, based on the 2000 revisions of the Chinese national ambient air quality standard and using the 2006–2007 monitoring station locations.

Ideally, officially reported data in the future should note the change in monitoring methodology on graphs showing data from both periods.

Impacts on Pollutant Concentration

As Mr. Andrews points out in the report, the Blue Sky Day metric is a “policy-relevant metric,” and, “an effective communication tool…to facilitate greater public understanding.” In other words, it is not a scientific metric, insofar as the cut-off point of API = 100 is rather arbitrary. Evaluating the effect of the aforementioned bias and monitoring station location change on reported vs. actual air quality requires analyzing pollutant concentrations.

Using a methodology to eliminate the reporting bias and normalize across similar reporting stations, Mr. Andrews’ ran a new concentration analysis for PM10 and generated the following results:

In 2006, an annual average PM10 concentration of 161 μg m−3 was reported, however, if the monitoring station used from 1984 to 2005 continued to be used in 2006, the concentration would be ∼167 μg m−3—an average concentration ∼6 μg m−3 higher than reported. In 2007, an annual average PM10 concentration of 149 μg m−3 was reported, however, if the original monitoring stations continued to be used in 2007, the concentration would be ∼161 μg m−3—an average concentration of ∼12 μg m−3 higher than reported.

Stated differently, he concludes that Beijing’s 2006 and 2007 reported values for PM10 were about 3.6% and 7.5% lower, respectively, than they would have been without data biasing or moving of monitoring stations. While this is concerning, the results are not nearly as dramatic as the difference in Blue Sky Days, as shown in Mr. Andrews’ report (Figure 3 from the report):

In the above graph, note that the difference in Blue Sky Days (shown as columns) is much greater than the difference in PM10 concentration (shown as red lines). Note also that while the trending on Blue Sky Days changes dramatically based on the new analysis (increasing 2001-2005, decreasing 2005-2007), the trending on PM10 does not show a large change under the new analysis.

Conclusions

As I mentioned earlier, Mr. Andrews’ result regarding the biasing of annual numbers of Blue Sky Days is powerful and dramatic. However, I’m not sure the result regarding 2006-2007 concentrations of PM10 is as dramatic, especially given the fact that PM10 has notoriously been one of the most difficult pollutants for Beijing to control. The Beijing EPB’s own data show a 2007 PM10 concentration of 149 um/m3, higher than 2003 and 2005. While adjusting the concentration data according to Mr. Andrews’ analysis may be important, doing so does not qualitatively change the 2001-2007 PM10 trends in Beijing.

Mr. Andrews concludes his report:

Although nine continuous years of air quality improvement has been reported in Beijing between 1998 and 2007, my analysis finds that these improvements, as indicated by the annual number of ‘Blue Sky’ days, are due to irregularities in the monitoring and reporting of air quality and not to less polluted air. Reported variations in air quality that occur as a result of changes in monitoring station locations or air quality standards, should be considered as inconsistencies in the metrics and not as actual changes in air quality.

While I agree with his analysis showing data biasing in the numbers of annual Blue Sky Days over the past few years, I think it is critical to clarify that such biasing does not mean that there was no improvement whatsoever in Beijing air quality over the last decade.