Monday, May 25, 2009

How Raw is Raw?

Dave, AKA as woox0LAVhIhtvEOQoeAC7D7Bm6_eesOdZg-- in a comment on the first Electra post said this:

Why don't you try again? Rather than dealing with co2science crap data, why don't you go to the actual, real raw daily data at the USHCN website like I suggested?

Now, I have been using CO2 science data because it is the real raw data, which they downloaded in 2007. Dave, wants me to go to another site, which he thinks is the real raw data. It isn't. If Dave would read the descriptive files that go along with what is given out at the particular site he likes he would find this statement:

The quality of the HCN data was
enhanced with the use of outlier and areal edits, and the data were
corrected for time of observation differences, instrument changes,
instrument moves, station relocations, and urbanization effects (Karl et
al. 1986; Karl and Williams 1987).

Now, what Dave is pointing us to is NOT the 'real raw' data as he says, it is the edited data that the USHCN folks claim is the original data. Unfortunately, Dave and I may have a different definition of what is raw data.

In fairness to Dave, One can argue that the data set offered doesn't have the following, also found in the descriptive files

The data have not been adjusted for station
relocations, heat island effects, instrument changes, or time of
observation biases. The nature of inhomogeneities arising from such
factors depends on a station's climatic regime.

Note that this second list doesn't say they didn't do the areal and outlier edit.

Back when I started looking into this, I found a statement somewhere that in about 2006 for the first time the USHCN allowed people to get the actual raw data--i.e. data without outlier editing and areal editing applied. I didn't realize the importance of that statement so I didn't capture it in my data base.

But the clear thing is that the data DAve says is the 'real raw data' isn't the real raw data but data edited to make it look pretty. The crap CO2 science data that he wants to ignore is apparently the real raw data before it has had a makeover by the make-up artists at the USHCN.

One of the reasons I am very suspicious of anything that smacks of editing by the make-up artists at the GISS is that they change history so easily, and history changes in just a 9 year period. The picture below is from Anthony Watts site but it shows clearly how editing is changing the past.

The sad thing is that the USHCN proclaims to give us raw data, but in fact they have already edited out all the lunacies in the data set, by their own admission. CO2 science may have the only really good look at the absolutely raw data set.

Now, if Dave or anyone else can locate on the internet the absolutely raw data from a USHCN or NOAA site, a dataset that has zero editing on it, I would be glad to stand corrected. Unfortunately the records are being continually updated at the USHCN site without anyone knowing what they are doing and what they are changing.


  1. So where does CO2Science get it's "ultra raw" USHCN data? I couldn't find their source on their webpage, other than the fact that they were using USHCN data.

    Personally if I'm going to look at at the raw data and the Oak Ridge National Lab site can't be counted on to provide that to me, then I am even less likely to resort to

    But as for the importance of "rawness" of the data, I think there's often a _reason_ the data is processed. In your work in geophysics, do you use completely raw unprocessed data in making your decisions?

  2. This comment has been removed by the author.

  3. They downloaded it in January 2007, but the files have been overwritten several times since then. I quizzed them about it.

    Now, why don't you look at the data that you say is what I should use? I posted on it this evening and plan other posts on it as well. Dave, the data is crap.

    Your question about the rawness of the data. No, we don't use absolutely raw data, although about 10 years before I got into the business, that was precisely what was used, and they found lots of big oil fields--they might have been more successful than this generation of oil folk simply because the only things they could see with that raw data were the huge fields.

    But, Hagiograph, one thing we don't allow are edits that so change the data as to turn them into something that they don't say. As far as I can tell, I haven't posted yet on the homogeneity filter/correction used by the 'scientists' at the USHCN. I put the word in quotes because when you take a raw measurement which says the station is cooling over time, and 'correct' it to say that it is warming over time, and then say you are measuring global warming, there is, in my not so humble opinion, a huge problem. They have taken the data and distorted it to make it say that which it doesn't say in its unedited form. We would NEVER EVER allow that in the oil industry. You would be fired for taking knowingly removing a well from a map, acting as if that structure had never been drilled before. Such behavior is not that of a true scientist. And I make that charge and will back it up, about the 'editing' of the data via the homogeneity correction.

    They are not treating the data honorably, Dave.

    Hagiograph, you can make whatever decision you want to make about what data you will use. I will use your data, as I did this evening to show that the data is so flawed as to be worthless to measure the tiny global warming signal. But no doubt, you haven't bothered actually looking at the data with a critical eye. You are a believer, nothing more.

    I don't follow people without checking things out. I have made a fortune finding a billion barrels of oil in places that a thousand other geophysicists have failed to find oil. They all looked where I looked, but they didn't find it. Why did I, along with my teams, find it and they didn't? Because of two reasons. Anyone who knows me knows that I don't give up easily and, I don't accept conventional wisdom. I question everything, everyone, every assumption. Until the data fits together like hands in a glove, I am not satisfied. Facts that don't fit must be explained or explained away in a coherent fashion. You are free to not look at any data you don't want to look at, but for petes sake, look hard with a skeptical eye at the data you think is good. It is horrible and isn't as good as you think it is.

    Wait till you see my post on Stillwater-Perry.