Archive for April, 2014

massive us embassy pm2.5 data dump

Tuesday, April 15th, 2014

Incredible! The State Department has released six years of hourly PM2.5 data from the US Embassy in Beijing. The data is actually wonderfully well-organized (kudos to whoever formatted it and prepared it for release!), and is just begging to be dug through. Here are some initial discoveries and thoughts based on a few hours of playing around with the data. (See also some good initial analysis from Wayne Ma/WSJ here and here.)

The Dataset
The dataset contains nearly 50,000 hourly readings from April 8, 2008, to March 31, 2014. There are some gaps in the coverage, but overall the data are fairly comprehensive. (It’s also clear that the reliability of the monitor – or maybe the reliability of maintenance – improved over time.) The data are exclusively hourly PM2.5 readings from the single monitor at the US Embassy. Because the data are in hourly concentrations of PM2.5, they must be processed into daily average concentrations and then converted to AQI. There are no ozone data, nor data from other monitors around the country or world. (Yet.)

The Best and the Worst
– Best hour: 9/28/2012 at 9am and 2pm: 0 ug/m3. Astonishing!
– Best day: 9/28/2012: 3 ug/m3
– Worst hour: 1/23/2012 at midnight: 994 ug/m3
– Worst day: 1/12/2013: 569 ug/m3
– Biggest single-hour change: 11pm to midnight, 1/22-23/2012: +769 ug/m3
– Longest streak of Unhealthy air (>150 ug/m3): 161 hours (2/19-26/2014)

Yearly Averages
The following figure shows Beijing’s yearly average PM2.5 levels according to the US Embassy monitor. Also shown on the figure are the US annual average PM2.5 standard (12 ug/m^3) and China’s (35 ug/m^3).

beijing yearly average pm25 2008-2014
The data, of course, show that Beijing’s air quality is atrocious, with annual average PM2.5 many times higher than recommended levels. But what about the trend? Well it’s actually not clear. Obviously the first quarter of 2014 has been terrible, but that could be just the season (see next graph). 2013 was slightly worse than 2012, but that’s not a trend; 2012 may have been abnormally clean. Overall, it looks like Beijing’s air has not gotten demonstrably worse or better over the past six years.

Monthly Averages
From these data we can take a first look at seasonal periodicity in PM2.5. This graph shows monthly average PM2.5 levels over the entire dataset (note: I should really show error bars in this, but don’t have the time to put that together right now and anyway this isn’t peer-reviewed, just a quick look):

beijing monthly average pm25 2008-2014
This is a strange pattern that I’m not sure how to interpret. It does seem to make intuitive sense that the winter months would be worse, but why also June and July? Perhaps the answer lies in the weather and wind patterns of the spring and fall that help disperse pollution, but that’s just a guess.

Let’s look at the entire time series (monthly averages):

beijing month average pm25 2008-2014
This is way less clean/clear. The extreme variability of the monthly air quality jumps out immediately, especially January 2013 and February 2014 (worst months), as well as January 2011 (best month).

Hourly Averages
Ever wonder how Beijing’s pollution changes over the course of the day? Well, take a look:

beijing hourly average pm25 2008-2014I’d say that looks like a pretty clear trend, with PM2.5 surging during evening rush hour and remaining high into the early hours of the morning. Is this linked to motor vehicle patterns (including trucks entering the city at night?) or related to diurnal weather patterns? I’m not sure yet, but certainly worth closer look and investigation.

Finally, at the risk of showing too much in one table, here’s the percentage binning of days by US AQI category in each of the years (further description of these bins is here).

2008-2014 beijing air quality by category
As also noted by Wayne Ma in WSJ, nearly half of the days are deemed “Unhealthy” by the US EPA. (I haven’t compared these yet against the Chinese scale, as Wayne did.) It’s clear that Beijing and the national government have a lot of work left to do. But really we knew that before we had 50,000 data points to prove it.

Next Steps
I get asked all the time by researchers if I have these data; I’m so glad that State has finally put them out there public. I’ve just scratched the surface here, and I’m really looking forward to seeing the further analysis people come up with. If I have time I’ll post some more analysis this week. Now if only the Chinese National Environmental  Monitoring Center would do a similar data dump…