« PIOMAS January 2016 | Main | Global sea ice area record minimum? »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Diablobanquisa

A couple of supplementary maps and comparisons:

New Walsh: August 1935 vs. September 1935:
https://diablobanquisa.files.wordpress.com/2016/02/35w.gif


If we forget the obvious inconsistencies on ice concentration and focus on the ice edge:
- The changes in the Siberian Seas look quite reasonable. In the Beaufort Sea, the change doesn't look impossible. Even the change at Greenland Sea isn't impossible.
- However, in the Barents Sea, I'd say the ice retreat looks excesive.
- Nevertheless, the largest and most striking changes are at Foxe, Baffin, Davis and Labrador. Just the areas without direct observations during August...

These are the maps that show the data sources used by Walsh for August (https://diablobanquisa.files.wordpress.com/2016/02/3508wsource.png ) and September (https://diablobanquisa.files.wordpress.com/2016/02/3509wsource.png )1935.

-White: AARI
-Blue: Kelly ice grids (the white area from DMI charts, even expanded)
-Light blue: ACSYS
-Yellow: DMI.
-Red: analog infilling.


Finally, September 1935, Walsh vs. ours:
https://diablobanquisa.files.wordpress.com/2016/02/35wcc.gif


Rob Dekker

Thanks Diablo,

Your analysis of the Aug 1935 Baffin Bay situation reveals something that is apparent over much of the Arctic picture from Aug 1935 :

It looks like Walsh essentially IGNORED the direct ice edge observations by DMI, and instead relied on something that resembles the Kelly fields, filled with an unrealistic ice concentration that smears low concentration out sometimes far to the north of the actual observed ice edge.

More evidence that this is what Walsh did is observed on the other side of Greenland :

North-West of Svalbard, DMI shows the ice edge clearly, with "densely packed" just north of the island (follow the 10deg longitude line) :
ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1935_08.jpg
Yet in the Walsh reconstruction, there is no clear ice edge along that line. Just a smeared-out patch with low ice concentration streching deep into the Arctic Basin, far towards the NP.
As if they ignored the ice edge observation from DMI altogether...

Same thing along the 40-50deg longitude slice (which crossed through Franz Joseph Land). There, DMI shows a clear, and observed ice edge with dense pack ice at about 77N lat, well south of Franz Joseph Land.
Yet Walsh shows open water all the way through Franz Joseph Land, and low concentration ice well north of the islands.
Miraculously, the ice south of Franz Joseph Land re-appears in September, likely because of direct observations with AARI.
Again, it looks like Walsh IGNORED DMI observations.

Also, I don't see ANY evidence that Walsh used ACSYS for Aug 1935.
After all, ACSYS sets the ice edge in the Nordic seas, and there is no evidence of ANY hard ice edge in Walsh reconstruction of the Nordic Seas in Aug 1935.

Just a smeared-out unnatural ice-concentration field that vaguely resembles the Kelly fields.

And I don't really want to jump to Sept 1935 just yet until we found out exactly how Walsh obtained Aug 1935, since for Sept 1935 90% of the Arctic is based upon source 21 :
"Analog filling of temporal gap"
Which is a joke if you 1) don't even acknowledge ice edge observations from Aug, and 2) the difference between Aug and Sept 1935 is an unprecedented 2 million km^2.
How did he do that ?

Rob Dekker

Needless to say, I now start to doubt not just Walsh' determination of the ice edge at various points in the record, but even his methods, which seem to rely on some form of "smearing" ice concentration over the Kelly fields, and an unexplained "temporal filling" procedure which creates unprecedented ice extent differences between Aug and Sept.

For 1935's spot-check, I would like to do the following to come to a better understanding of the actual extent at the time :
- Determine the ice edge in Aug 1935 along all lines of longitude, using ALL available data (AARI, DMI, ACSYS) and ignoring the Kelly field.
- Determine the ice extent in Aug 1935 based on that observed data
- Determine the ice extent in Sept 1935 by subtracting a reasonable difference that is sustained by the typical Aug-Sept difference from the satellite era (which is something like 300 k km^2).

I have work to do.

Bill Fothergill

@ Diablo & Rob

Gents, this is fascinating stuff. Before I gave up on WTFUWT as a total waste of time, I remember trying to discuss the Walsh dataset, but it was like trying to talk to petulant spoiled brats. It's great to see it being analysed at a level that far exceeds my meagre abilities, and seeing constructive - as opposed to destructive - criticism at work.

However, if I may drag you somewhat forward in time to 1990, and the 1st Assessment Report. There appears to be something very weird going on with the NATICE data, but you gents may have already have sorted it out.

The publicly available data is constructed in bi-weekly blocks, but there appear to be frequently occurring partial data-drops which can manifest in the first fortnight of the year. For example, take the following years ...

1974, 75, 76, 77, 81, 82, 83, 86 & 90

In each instance, the averaged ice levels are considerably down on the closing data values of the previous year.

On average, those 9 early-January data points show a drop of 1.68 million sq kms from the previous year's closing value. Similarly, the data points covering the second half of January in each of those years show an average jump of 2.91 million sq kms over the preceding fortnight.

In February of 2014, I emailed NAVICECEN and asked if there was any explanation/hypothesis for the anomalous data, but I never had any response.

Perhaps you guys can shed some light on this?

Also, is there a flat file somewhere (perhaps in Excel, PDF or CSV format) which contains this data? I had to manually transcribe, and therefore - given my deteriorating eyesight - am slightly concerned about transcription errors.

Diablobanquisa

For 1935's spot-check, I would like to do the following to come to a better understanding of the actual extent at the time :
- Determine the ice edge in Aug 1935 along all lines of longitude, using ALL available data (AARI, DMI, ACSYS) and ignoring the Kelly field.
- Determine the ice extent in Aug 1935 based on that observed data
- Determine the ice extent in Sept 1935 by subtracting a reasonable difference that is sustained by the typical Aug-Sept difference from the satellite era (which is something like 300 k km^2).

I have work to do.

Excellent, Rob. As you stated, I suggest focusing on the sea ice edge and ignoring the (non consistent) ice concentrations.

A couple of points:

- The mean difference during 1981-2010 between August and September values is about 0.7 M km^2. (the largest about 1 M km^2 while the smallest about 0.2-0.3 M km^2)

- Maybe you should also compare your Sept results with AARI direct observations for that month.

Cheers

Diablobanquisa

Hi Bill,

If I've understood correctly, you were refering to this: http://www.natice.noaa.gov/products/ice_extent_graphs/arctic_weekly_ice_extent.html

I don't know which is the reason for those spurious drops and jumps, because the charts the graph is derived from don't show any anomalous jump. For instance, look at 1975/12/30 and 1976/01/06 charts:
- ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02172/gifs_weekly/nic_weekly_1975_12_30_tot.v0.gif
- ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02172/gifs_weekly/nic_weekly_1976_01_06_tot.v0.gif

As far as I know, unfortunately there isn't any flat file which contains this data.

Cheers

Diablobanquisa

Rob, I have been reading again the documentation of the new Walsh dataset (https://nsidc.org/data/docs/noaa/g10010-sea-ice-1850-onward/G10010_SIBT1850.pdf ).

They didn't use directly DMI maps (https://nsidc.org/data/docs/noaa/g02203-dmi/ ), but concentration fields derived from those DMI maps: http://nsidc.org/data/docs/noaa/g10007-dmi-seaice/

For instance, for August 1935 they used this map: ftp://sidads.colorado.edu/DATASETS/NOAA/G10007/browse-images/DMI_1935_08.jpg (instead of the original DMI one: ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1935_08.jpg ).

They only used the red areas of the former. I guess they used this dataset instead of original DMI charts because it's already gridded and georeferenced; of course, you need gridded data in order to do this kind of work.

And the use of this dataset can help explain what happens south of Franz Joseph Land: no red area -> Walsh didn't use DMI data there.

Around Franz Joseph Land Walsh used ACSYS sea ice edge (August 26th): ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02169/ice_edge_positions/browse/aug/iep19350826.jpg

It's worth mentioning that this ACSYS chart and the DMI map disagree. Which is right? Or are they observations from different date? Or what is wrong?

Let's look at the area south of Franz Joseph Land according to this AARI map for September 1st (observations from September 1st to September 10th): ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02176/pngs/1935/w19350901_tc.v0.png

The ice that you said that miraculously re-appeared comes from this map. However, that ice could continue further east at that latitude, in agreement with DMI, or could be the easternmost section of it, thus in agreement with ACSYS.

I think that the open water limits as showed by the AARI chart suggest that the sea ice edge should be where DMI shows it. But, unfortunately, although we can be sure that the blue area in the AARI map was open water, we can't be so sure whether the ice edge was just at the latitude and longitude where open water ends or not. It could also be the limit of the aircraft reconnoissance flight, and the open water could continue further north or west... Both interpretations are plausible...

In fact, ACSYS charts also use AARI data for the Barents Sea, so I guess that ACSYS ice edge near Franz Joseph Land could be derived from the AARI chart linked above, and that the norwegians interpreted AARI data according to the latter option (open water ends where the flight ends).

Curiously, during 1930s DMI also used information provided by the Soviets, so I guess the danish could also be using the same AARI data, but they interpreted it in the opposite way (open water ends just where the sea ice edge is).

Nothing is easy before the satellites, even when we have data sources...


Diablobanquisa

Regarding Kelly grids:

As far as I know, those grids aren't publicly available. As they are supposedly derived from DMI charts, they should be similar to the sum of blue and red areas of this map: ftp://sidads.colorado.edu/DATASETS/NOAA/G10007/browse-images/DMI_1935_08.jpg

However, Kelly grids seem to have expanded the sea ice covered areas, as we saw at Baffin, Davis and Labrador.

Kelly field for August 1935 should be similar to this map (HadISST1, August 1935, https://diablobanquisa.files.wordpress.com/2016/03/kelly_hadisst_3508.png ), where we can see the extra ice 'created' by Kelly at Baffin, Davis and Labrador.

(at Barents, Kara and the Siberian coast, it seems that Kelly also 'created' more extra ice).

Rob Dekker

Diablo said :


They didn't use directly DMI maps (https://nsidc.org/data/docs/noaa/g02203-dmi/ ), but concentration fields derived from those DMI maps: http://nsidc.org/data/docs/noaa/g10007-dmi-seaice/

For instance, for August 1935 they used this map: ftp://sidads.colorado.edu/DATASETS/NOAA/G10007/browse-images/DMI_1935_08.jpg


It makes sense that they used these gridded maps, but I'm not sure if they used the "concentration" part of it.
The "red" direct observations of high concentration ice in that gridded product result in only "low concentration" ice in the Walsh reconstruction. I only need to point out that Baffin Bay issue, but it is all over that map : DMI observations are interpreted as low-concentration in Walsh' reconstruction along the east side of Greenland, NW of Svalbard, South of Svalbard, and yes also south of Franz Josef Land.

Rob Dekker

Looking at these 1935 DMI maps and the data from AARI, I wonder if 1935 was one of these years with and extreme "Polar Express".
It seems that the low extent on the Russian side is made up by high extent in the Greenland sea and the Barents.

Rob Dekker

Diablo said


Kelly field for August 1935 should be similar to this map (HadISST1, August 1935, https://diablobanquisa.files.wordpress.com/2016/03/kelly_hadisst_3508.png ), where we can see the extra ice 'created' by Kelly at Baffin, Davis and Labrador.

Yes. But HadISST1 projected some 11 M km^2 for July
http://www.star.nesdis.noaa.gov/star/documents/meetings/Ice2015/dayThree/1_Walsh_J_dayThree.pdf

which may still be true, but Walsh does not address that.
Instead, he smeared low concentration all over the Aug map, including areas that have defined ice observations.
That is simply not right.

Diablobanquisa

I realized something odd regarding Kelly fields.

I think they are derived from the white area of DMI charts... of the previous month.

August 1935, Kelly field (using HadISST as a mirror of Kelly fields)): https://diablobanquisa.files.wordpress.com/2016/03/kelly_hadisst_3508.png

July 1935, DMI map: ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1935_07.jpg

The Kelly field for August matches pretty closely the white coloured area of DMI's July map.

Look at the white area at Baffin-Davis-Labrador on the new Walsh's map for August 1935 (they state that the source for that region are Kelly fields): https://diablobanquisa.files.wordpress.com/2016/03/1935_08_w.png

So in fact Walsh et al. are including on their August map data derived from DMI's July map...

Diablobanquisa

Look at the white sea ice covered area at Baffin-Davis-Labrador on the new Walsh's map for August 1935

Rob Dekker

Diablo,
You may be onto something.

There is indeed a striking resemblance between the DMI July 1935 charts and the Walsh Aug 1935 reconstruction.
I agree that it looks like the July Kelly fields were mistakenly placed in August...

However it is strange that the resemblance stops outside of the greater Baffin Bay area.

Bill Fothergill

"Hi Bill,

If I've understood correctly, you were referring to this: "

http://www.natice.noaa.gov/products/ice_extent_graphs/arctic_weekly_ice_extent.html

Yep, that's precisely what I was looking at. The sheer scale of the data drops only really become obvious when one sets the start and stop years to the same value. When only a single year is selected, rather than a range of years, the values associated with each data point are displayed on the graph.

For example, the final value in December 1974 was 13.21 million sq kms. If one then steps the chart forward to 1975, the opening two January values were 10.59 and then 14.75 - which is obviously ludicrous.

The credits on the 1990 First Assessment Report certainly suggest that NAVICECEN (or its incarnation at the time of writing) was the source for Graph 7.20(a) which purported to show the trajectory of Arctic sea ice over from the early 1970's to 1990. I would have liked it better had I been able to reconstruct their graph myself.

As regards the detail behind the NAVICECEN material, I have looked also at the NSIDC merged file which splices ESMR, SMMR and SSMI data ...
ftp://sidads.colorado.edu/pub/DATASETS/nsidc0192_seaice_trends_climo/total-ice-area-extent/esmr-smmr-ssmi-merged/gsfc.nasateam.extent.1972-2002.n

There is no sign of the anomalous behaviour seen in the NAVICECEN charts.

You may, or may not, be aware that the National Maritime Museum in London held an exhibition on WWII Arctic Convoys about 5 years ago. At one stage, there was a hyperlink to a listing of all the Merchant Navy ships involved - and their logs.

Naval logs should be something of a goldmine, as they would show, amongst other things: Date/Time of observation, Latitude, Longitude, prevailing conditions, ice status....

Sadly, this link now produces a 404 Error Message. However, someone from the Museum might be able to help out. Their contact form is available here...
http://www.rmg.co.uk/work-services/contact/enquiries-form

Diablobanquisa

However it is strange that the resemblance stops outside of the greater Baffin Bay area.

Yes, I'd say that's because they derived the sea ice edge at other regions (Greenland Sea, Barents, Siberian coast) from other sources (DMI fields, ACSYS, AARI). It seems Walsh et al. used Kelly field (blue) at the grid cells where no other data source was available: https://diablobanquisa.files.wordpress.com/2016/02/3508wsource.png

So, I think they used the Kelly field to derive the sea ice edge at the 'great Baffin', CAA, Chukchi and Beaufort (except for the direct observations along the Alaskan coast, derived from ftp://sidads.colorado.edu/DATASETS/NOAA/G10007/browse-images/DMI_1935_08.jpg)

However, outside the 'great Baffin' it doesn't matter because Kelly fields (and the white areas of DMI maps they are derived from) for July and August 1935 from CAA to Chukchi Sea are almost identical (ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1935_07.jpg and ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1935_08.jpg ) However, it seems there is a small difference at Chukchi Sea. In August, the sea ice edge goes parallel to the coast between Icy Cape and Cape Lisburne, whereas in July the sea ice edge draws away from the coast, moving westward. Looking at Walsh's August chart (http://i1272.photobucket.com/albums/y396/RobDekker/Aug_1935_zpsrmg1mdfz.png) I'd say the sea ice edge draws away from the coast, thus corresponging to DMI's July map (August Kelly field).

Diablobanquisa

Instead, he smeared low concentration all over the Aug map, including areas that have defined ice observations.
That is simply not right.

I agree. I don't understand how Walsh et al. got their concentration values, and I don't think those concentrations are reliable neither consistent. So, I'd focus on sea ice edge, ignoring concentration.

Diablobanquisa

I agree that it looks like the July Kelly fields were mistakenly placed in August...

I've been checking August Walsh's maps against July and August DMI ones during 1935-1939 and 1946-1952. Every year, Walsh (through Kelly fields)has included data from July DMI maps.

So, at Baffin-Davis-Labrador during August 1935-1939 and 1946-1952 Walsh's extent looks overestimated by several hundred M km^2.

(It seems that some years the interference from July data affects not only Baffin but also other regions, particularly Chukchi and Bering)

Diablobanquisa

In August 1939, this issue affects also Barents Sea.

Diablobanquisa

looks overestimated by several hundred M km^2.

I meant hundred K km^2

Rob Dekker

Diablo said

I don't think those concentrations are reliable neither consistent. So, I'd focus on sea ice edge, ignoring concentration.

I agree. In the Walsh dataset, we need to focus on ice 'extent', and ignore 'area'.

So the first thing I did was to visualize the Walsh data in such a way the ice edge for extent (>15% ice concentration) is clearly visible.
I did that here, for Aug 1935 :
http://i1272.photobucket.com/albums/y396/RobDekker/seaice_Aug_1935_zpsh6r5qn9o.png

I don't have a nice picture of 450pixels max (which would fit in typepad's picture width, so please forgive me; you need to click the link to see ice extent in Aug 1935.

Note that in this picture, it seems clear what Diablo already pointed out before : Baffin Bay are seems to contain much more ice than the DMI charts for Aug 1935 suggest, and the other side of Greenland (the Greenland sea and the Barents sea specifically) seem to contain a bit less than what DMI suggests, especially the Southern tip of Greenland and the area south of Franz Joseph land. The Russian side of Walsh' data set seems to be consistent with Diablo's findings.

Second, I wrote a small program to analyze the gridded data matrix from the Walsh data set, and calculate ice extent.

As-is, ice extent for this Walsh reconstruction of Aug 1935 comes in at 8.977 M km^2.

That 8.977 number is consistent with Walsh' findings in fig 1 of the Walsh data set documentation. That gives me hope that I did the grid cell area calculations correctly. I assumed a pure spherical Earth, but it should be right within 2 or 3 decimals.

Now, as Diablo argues, DMI suggests that the ice extent in Baffin Bay was much less than Walsh assumed. In fact, it looks like Walsh messed up the months, and Baffin Bay picture may be from July 1935.

So I applied a filter.
If I take out the box from 30-75 N, 50-70 W, which covers ALL of Baffin Bay and south of that as well as most Hudson Bay ice, extent lowers to 8.479 M km^2.

The absolute lower bound for Aug 1935 is when we take out the box 30-75 N, 50-85 W, which means all of Baffin Bay, Hudson Bay and a part of the Canadian Arctic.
Taking out that box results in extent of 8.242 M km^2.

The reason I say this is that absolute minimum is because we know there WAS ice in Baffin Bay (DMI charts) AND we know that the Walsh reconstruction for 1935 is already low for the Nordic Seas (especially the tip of Greenland and the area south of Franz Joseph Land).

So here we go for MY first estimate : I believe, based on all the evidence presented above, that in Aug 1935 was around 8.5 M km^2.
If we deduct the 'average' sea ice extent drop of about 700 k km^2
between Aug and September, then Sept 1935 will have been about 7.8 M km^2.

More later

Diablobanquisa

Thank you Rob, I think you have done an excellent work.

However, I'd like to highlight another issue concerning Walsh's map (and thus your estimated extent):

What about the easternmost section of Beaufort Sea (from Mackenzie delta to Banks Island), Amundsen Gulf, Dolphin and Union Strait and Coronation Gulf?

I'd say that whole region is about 0.3 M km^2 or so. Walsh et al. are assuming it is completely ice covered relying on Kelly fields (derived from DMI maps, although at this region it doesn't matter whether they are using July or August DMI maps because the white coloured area is identical for both of them).

However, we don't have any direct observation for that region. Nevertheless, usually most of that region is open water in August (look at August NSIDC maps from 1979, for instance this for 1988: ftp://sidads.colorado.edu//DATASETS/NOAA/G02135/Aug/N_198808_extn.png or at the climatologies created by Environment Canada: http://iceweb1.cis.ec.gc.ca/30Atlas/page1.xhtml?region=WA&lang=en , even though concentration and extent values calculated by Environment Canada are systematically higher than those derived from passive microwave data).

So, since your calculation assumes that region as completely ice covered, it assumes the maximum possible extent there. The extent at that region can't be higher, but it could be lower (in fact, climatological data suggest most of that region should be open water in August). So, I think your 8.5 estimate is the absolute maximum. Taking into account this 'Mackenzie-Coronation' issue, I'd say your estimate has an error band of about 0.3 M km^2 that moves only downwards.

Based on all the stated above , I'd say the 'most likely' estimate would be around 8.3 instead of 8.5 M km^2. (so the Sept estimate would be around 7.6 instead of 7.8).

I think there still are other issues that could move the estimate slightly further downwards, but I will discuss those later.

Cheers

Diablobanquisa

Other areas at CAA could also be open water (or not). For instance, Queen Maud or St Roch and Rasmussen Basins. The effect on extent numbers would be very slight because they are small regions, but the fact is that any error at CAA or easternmost Beaufort would necessarily lead to a lower extent, as Walsh shows CAA and Beaufort completely ice covered.


Concerning the Siberian Sector, look at this August 20th AARI map: ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02176/pngs/1935/e19350820_tc.v0.png
There are two straight lines, one from the north of Taymir to New Siberian Islands, and the other one from NSI to Kolyma river or so. However, AARI map doesn't show a defined ice edge. We can be sure that until those straight lines it was open water, but we can't be sure if the sea ice edge was just at those lines or further north.

Walsh has created an ice edge just at those lines by using Kelly field 'below' the other sources (here, AARI). The sea ice edge can't be further south of the two straight lines (if we trust AARI data) but it could be further north.

I think the sea ice edge at the Siberian Sector should be quite close to the straight lines and thus to the sea ice edge on Walsh's map. But, again, any error would necessarily lead to a lower extent.

So, I'd agree with an August extent around 8.3 M km^2 (or even 8.5), but I understand those numbers as the absolute maximum, the upper limit.

(related to the 'Beaufort-CAA' issue, when analyzing 1952 we'll have the chance to talk about the overall reliability and internal and external consistency of DMI white coloured areas)

Thanks again for your work, Rob.

Cheers

Rob Dekker

Thanks Diablo.
Regarding that eastern side of the Beaufort, I don't think there is evidence that it was ice free in Aug 1935.

Remember that the climatologies by Environment Canada report 1981-2010, which was a period where Arctic sea ice extent was barely ever above 5 M km^2. And for the 1979 case that you mention, the Beaufort was low on ice, while the Siberian Sea was high.
In 1935, it appears to have been the other way around.

But even more specific, notice that DMI reports observations of dense sea ice along the Beaufort shores, all the way to the Mackenzie river delta. That suggests the situation in the Beaufort in 1935 was nothing like it was in 1979.

Walsh reports 30-45% ice concentration for that eastern-most area of the Beaufort, and I think that is reasonable considering the direct DMI ice observations next door.

Also, I don't think that that 8.5 number is an 'upper' bound. It already excludes ANY ice in Baffin Bay, where we know there was some, and it also excludes ice south of Franz Joseph Land, where DMI observed ice in Aug and AARI in September.

I don't think we can get more accurate than 8.5 M km^2 for Aug 1935 unless we fill-in ice observations grid-point by gridpoint and then find 2D "analogs" for spacially and temporally distant points.
Incidentally, such a method would also gain a "uncertainty margin", which is another thing I miss in the Walsh analysis.

Diablobanquisa

Thanks, Rob.

Regarding that eastern side of the Beaufort, I don't think there is evidence that it was ice free in Aug 1935.

I agree, there is no evidence of that. But the only evidence that it was ice covered is that is white coloured on DMI map.

Are the white coloured areas reliable and consistent? Can we compare them with the direct observations from 1953 and with the passive microwave direct observations from 1978? (we'll discuss this again later).


Remember that the climatologies by Environment Canada report 1981-2010, which was a period where Arctic sea ice extent was barely ever above 5 M km^2.

1981-2010 August mean extent is 7.22 according to NSIDC.(6.5 in September).


But even more specific, notice that DMI reports observations of dense sea ice along the Beaufort shores, all the way to the Mackenzie river delta. That suggests the situation in the Beaufort in 1935 was nothing like it was in 1979.

Rob, I specifically linked to August 1988 map. If you look at it... ftp://sidads.colorado.edu//DATASETS/NOAA/G02135/Aug/N_198808_extn.png
... you will see that the ice along the Beaufort shore all the way to the Mackenzie delta is exactly as in 1935. But the easternmost Beaufort Sea, Amundsen Gulf, Coronation Gulf... are open water.


It already excludes ANY ice in Baffin Bay, where we know there was some, and it also excludes ice south of Franz Joseph Land, where DMI observed ice in Aug and AARI in September.

Well, if I understood correctly, your 8.5 estimate assumes about 0.25 M km^2 at the box 30-75 N, 50-85 W. Taking into account the direct observations of the DMI map, it looks reasonable, including Baffin Bay.

The fact about the Franz Joseph Land issue is that DMI and ACSYS disagree, and that the judge (AARI) comes already in September and it's quite ambiguous. Even though DMI was correct, the difference would be around 0.10 or 0.15 M ?

It's a fact that in CAA and Beaufort the extent can't be higher than your estimate, it's simply impossible. But it could perfectly be lower, any error neccesarily would lower extent.

If we rely on the white coloured areas of DMI charts... how can we compare 1935 with 1940-1945 when DMI maps are not available?

If the white coloured areas are proven wrong at any point, should we reconsider their use in 1935?

If we are going to trust DMI white coloured areas, shouldn't we test their reliability and internal and external consistency? Unless we do it, can we extract any conclusion about trends relying on DMI white areas and comparing them with direct observations during other years?

Diablobanquisa

I trust DMI direct observations, ACSYS direct observations and AARI direct observations.
But I don't trust the white coloured areas on DMI maps, I don't think they are reliable nor consistent, and in my opinion constructing long term time series is everything about consistency.

Look at August 1952 DMI chart: ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1952_08.jpg
The white coloured area suggest an extent well over 9 or even 10 M km^2.

However, look at August 1952 AARI charts for the eastern Russian sector: ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02176/pngs/1952/E19520811_tc.v0.png and for the western Russian sector: ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02176/pngs/1952/W19520915_tc.v0.png

If we trust AARI direct observations, DMI's white coloured area is plainly wrong and completely unreliable for the whole Siberian sector.

If we trust ACSYS: ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02169/ice_edge_positions/browse/aug/iep19520826.jpg , DMI's white coloured area at Svalbard and Barents is also wrong.

If the white coloured area is completely wrong and unreliable from Svalbard to Bering Strait, why should it be reliable at the American sector?
Even if you think it is not reliable at the Siberian Sector, but it is reliable at the American sector, this implies it is not consistent.
And, if you think it is reliable in 1935 but it's not reliable in 1952, that also implies a lack of consistency.


And yes, I think that the analog approach you suggest would be better than relying on DMI's white coloured areas.

(regarding Mackenzie-Coronation, that area is mostly open water in August not only during the satellite era, but also before. For instance, we can look at this map for August 1970 from the Canadian Ice Service: https://diablobanquisa.files.wordpress.com/2016/03/sc_a10_19690815_xxxxxx.gif from CIS Archive: http://iceweb1.cis.ec.gc.ca/Archive/page3.xhtml )

Diablobanquisa

Sorry, I linked 1969 instead of 1970: http://ice-glaces.ec.gc.ca//www_archive/AOI_10/Charts/sc_a10_19700819_XXXXXX.gif


Diablobanquisa

I don't think we can get more accurate than 8.5 M km^2 for Aug 1935 unless we fill-in ice observations grid-point by gridpoint and then find 2D "analogs" for spacially and temporally distant points.
Incidentally, such a method would also gain a "uncertainty margin", which is another thing I miss in the Walsh analysis.

I mostly agree: using the available data sources, 8.5 is a good estimate (but the question is whether all the data sources are reliable and consistent enough, and I think 'Kelly fields-DMI white areas' aren't, because they aren't derived from any direct observation).

I think that the analog approach is a good method, even though it has an uncertainty margin, as you stated.

The analog approach is similar to Walsh et al's methodology, and I think it is correct: they fill the grid cells with direct observations where available, and infill the grid cells where direct observations are not available using the analog approach.

However, the issue is that Walsh has used Kelly fields as a valid data source (at the grid cells where no other data is available). I think Kelly fields are unreliable, so in my opinion it would had been better to infill those grid cells using the analog approach instead of Kelly fields.

I guess that if Walsh et al. had used the analog approach for the grid cells they infilled with Kelly data in August, their August results would have been much more consistent with their September results.

So, I'd suggest:

- for August 1935 : available direct observations (DMI, ACSYS and AARI) + analog approach (instead of direct observations + Kelly field, as Walsh has done).

- for September 1935: available direct observations (AARI) + analog approach (just what Walsh has done).

I'd suggest the same approach to every August and September during 1935-1952: all the available direct observations for each month + analog approach (so, exluding 'Kelly fields-DMI white areas').

Diablobanquisa

However, look at August 1952 AARI charts for the eastern Russian sector: ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02176/pngs/1952/E19520811_tc.v0.png and for the western Russian sector: ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02176/pngs/1952/W19520915_tc.v0.png
ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02176/pngs/1952/W19520812_tc.v0.png

Rob Dekker

Hi Diablo,

Sorry for the mistakes in my latest posts (I meant 8M km^2 instead of 5, and I misread 1988 for 1979).

I'm all with you regarding the white areas (or Kelly fields) on the DMI charts. These are only educated guesses and not real observations. So we should NOT use them unless it's absolutely necessary.

For spatial fill-in, I put far more trust in a method that uses "analogs" from the satellite era.

For example, regarding the situation around the East Beaufort/Mackenzie-Coronation area in Aug 1935, the Aug 1988 map you linked to serves as a good "analog". After all, it matches the nearby ice observations from the Aug 1935 DMI map.

I imagine we can develop a method that uses "analogs" to determine spatially separated areas at any one time.
For example, if there is an ice observation at any particular location (x,y) at any one time (t), we could use the satellite era (1979-1015) images during the meting season, to find ONE image per year that matches that ice edge observation exactly. That would generate some 36 ice extent images that match with that (x,y) ice edge location. The ice extent in all other locations of the Arctic in these 36 images then serves as a guide on where the ice edge could have been elsewhere in the Arctic at time t, and also we will can calculate an uncertainty margin for the ice edge at each location as well as an uncertainty margin for the ice extent at time t.

We can add more constraints to each location if there are multiple ice edge observations at the same time at places separated from (x,y), and for observations at different times temporally separated from t. That can be done by standard statistical methods.

That is the sort of method that I'm thinking of that would formalize the process of determining ice extent at any time and place given ice edge observations at certain times and places.

Such a method would be far more formalized and accurate than relying on a "climatology" or the "white" areas on DMI graphs or the derived Kelly fields.

Especially since Walsh seems to have misplaced the Kelly fields by a month at least in some areas of the Arctic.

Rob Dekker

Semi-related, three comments/questions :
1) Typepad seems to cut-off ftp links, so they do not resolve.
2) sidads.colorado.edu links seem to have dead links (to NOAA) in them, so we cannot click through. See this base link, for example :
ftp://sidads.colorado.edu//DATASETS/NOAA/G02135
The months under this ftp link point to ftp://datasets/NOAA which is pointing into non-existing space.
3) Do you know if there is an algorithm that translates the EASE grid to Walsh's 1/4deg x 1/4deg grid ? That translation would be very helpful in running some experiments with the method of ice-edge-based spacial filling that I described above,

Rob Dekker

In short, I don't want to argue much about if Aug 1935 was 8.3 or 8.5 M km^2 nor if that was an upper or lower bound.
I'm more interested in developing a method that is statistically sound, and follows a formal algorithm using actual ice observations as input.

Diablobanquisa

Thank you, Rob.
I agree, and the analog approach you suggest sounds really well.
Regarding EASE Grid, we translated AARI data to 1deg x 1deg grid to match HadISST. I guess we could change the code in order to tranlate EASE Grid to 1/4deg x 1/4deg grid, but I should ask to my co-author.
Cheers

Diablobanquisa

Rob, which is the EASE Grid dataset you'd like to translate to 0.25x0.25 deg.?
AARI?
Or satellite data? Which is the satellite dataset you are planning to use? Maybe you could use http://nsidc.org/data/docs/daac/nsidc0051_gsfc_seaice.gd.html (it's polar stereographic, and it's the dataset used by NSIDC to get their popular Sea Ice Index numbers (https://nsidc.org/data/docs/noaa/g02135_seaice_index/ ) ).

(I'm afraid I haven't understood what is wrong with ftp://sidads.colorado.edu//DATASETS/NOAA/G02135 . When I click on the months under this ftp link, I get access without any problem...)

Rob Dekker

Thanks Diablo,
Did not have much time today.

But regarding the broken links on sidads, it seems to be a problem only on Chrome. I tried IE and Safari, which all work correctly.

Regarding grid translations, I'm looking for some grid where we can merge both satellite observations (such as G02135) with ice observations (such as AARI, and DMI-gridded), and possibly Walsh's dataset (although we already know that that one is tainted by Kelly fields).

Which grid would you suggest ?

Diablobanquisa

Hi Rob, I'm not sure, I think it also depends on your skills and the software or language you are planning to use.

Anyway, maybe the most straightforward approach could be using the Walsh netcdf file itself.

I think that if you remove (by setting to NAN) all the grid cells on seaice_conc that have a 4 (Kelly fields) on seaice_source, you will get the sea ice edge as defined by DMI, ACSYS and AARI direct observations, without any interference from Kelly fields.

Regarding the satellite era, every data on the Walsh dataset from 1979 onwards are satellite data, so you could directly use the Walsh dataset to look for analogs during the satellite era in order to fill the removed grid cells.

(I'd also suggest to set every grid cell with ice to a given value, for instance 100, in order to get extent data instead of concentration/area, because I think that the concentrations derived from different sources won't be consistent between them)

Rob Dekker

Thanks Diablo, these are great tips.
I'll be working on an "analog" method for spatial (and possibly temporal filling) using the Walsh data set excluding the Kelly fields, using Walsh' 1/4x1/4 grid.
Meanwhile, I'd like to spot-check 1952 and 1963.
This is starting to be fun :o)

Rob Dekker

I'm working on an unbiased "analog" recognition algorithm for spatial filling, but meanwhile I took a first closer look at 1952.

That year is interesting, because both in Diablo's reconstruction (based on AARI) and Walsh' reconstruction, Sept 1952 comes out really low. Walsh extent graph clocks it at 6.5 M km^2, lower than any year in the 20th century, at least until the 90's.

When we take a closer look at the origin of this number, Walsh' data set shows that most of the extent in Arctic in Sept 1952 comes from source 20, which is "analog filling of spacial gaps".

In fact the only observations in Sept 1952 are from the Russian side of the Arctic, from AARI, which explains why Diablo and Walsh are in such tight agreement about that month.

The problem with this ultra-low Sept 1952 number becomes clear once we look at Aug 1952, which is sustained by more ice observations (such as DMI and ACSYS). I calculated the Aug 1952 ice extent from the Walsh data set directly, and obtained 9.27 M km^2 for Aug. Here is the image which shows ice extent (>15% concentration) quite clearly :

Now, it seems to me that this situation in Aug 1952 is fairly similar to Aug 1935 : On the Russian side, ice is fairly low, but on the Atlantic side (especially the Greenland sea) as well as the Canadian Arctic, ice extent is high.

And, as opposed to 1935, where we could still argue that Walsh followed the (wrong) Kelly fields, this time DMI observations kind of match with this high extent on the opposite side of the Russian Arctic.

You could still argue that this image shows somewhat higher extent in the Bering and Baffin Bay than what DMI direct observations reflect, but not much, and certainly not as much as 1935.

So I would argue that the extremely low ice extent in Sept 1952 in BOTH Diablo's and Walsh's reconstructions is probably an artifact of choice of climatology for that month, and based on the Aug 1952 ice extent of 9.27, we would (based on the standard 700 k km^2 decline between Aug and Sept) expect Sept 1952 to end have been around 8.5 M km^2.

Diablobanquisa

I don't agree, I think that the high extent in August 1952 on Walsh's dataset is an artifact derived from the use of Kelly fields (that have been misplaced and, anyway, are completely unreliable and non consistent).

Firstly, Walsh is using again Kelly fields derived from DMI's... July map. Look at Labrador, Bering... and compare with DMI's July map: ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1952_07.jpg

But that's not the only problem. Even though they were using August instead of July, they would be still relying on an unreliable and non consistent source: Kelly fields, derived from DMI's white coloured areas.

As I stated above, we have direct observations for August 1952: AARI charts ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02176/pngs/1952/E19520811_tc.v0.and ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02176/pngs/1952/W19520812_tc.v0.png and ACSYS map: ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02169/ice_edge_positions/browse/aug/iep19520826.jpg.

If we trust AARI direct observations and AARI direct observations, DMI's white coloured area is plainly wrong and completely unreliable.

If the white coloured area is completely wrong and unreliable from Svalbard to Bering Strait, why should it be reliable at the American sector, where we don't have any direct observation?

I don't think we can't reach any reliable conclusion about August 1952 relying on the white coloured area on DMI's map. I'll wait until you finish your analog algorithm.

Cheers


Diablobanquisa

You could still argue that this image shows somewhat higher extent in the Bering and Baffin Bay than what DMI direct observations reflect, but not much, and certainly not as much as 1935.

Rob, DMI doesn't present any direct observation at Bering, and the direct observations at Baffin are just a small patch of low concentration sea ice near Melville Bay. This is far from being enough to infer any reliable ice edge for the whole region (Baffin-Davis-Hudson Strait-Labrador-Foxe).


The problem with this ultra-low Sept 1952 number becomes clear once we look at Aug 1952, which is sustained by more ice observations (such as DMI and ACSYS).

Sources for August 1952 (white: AARI direct observations; light blue: ACSYS direct observations; yellow: DMI direct observations; blue: Kelly field, areas without any direct observation: https://diablobanquisa.files.wordpress.com/2016/03/seaice_source-in-g10010_sibt1850_v1-1.png

The sea ice edge at Baffin-Davis-Hudson Strait-Labrador, Foxe Basin, almost whole CAA, whole Beaufort, eastern Chukchi and Bering is completely derived from Kelly fields (white coloured area on DMI's map).

Is the August 1952 Kelly field reliable? Well, where we have direct observations to compare against (ACSYS at Svalbard-Barents, AARI from Kara to western Chukchi), it results that Kelly field (DMI's white coloured area) is completely wrong, and it strongly overestimates sea ice extent. Why should it be different at Baffin, CAA, Beaufort, Chukchi...?

So, I think that you can't extract any conclusion about how September 1952 could and could not have been, relying on the white coloured area on August 1952 DMI's map: ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1952_08.jpg

Rob Dekker

Diablo, I don't even disagree with you.

Walsh's reconstruction of August relies much on the Kelly fields (at least for the pre-1956 period that Kelly fields are available) , and it certainly does not help that Walsh seems to have misplaced these fields by a month either.

Also, his September analysis seems to rely much (if not only) on AARI measurements, just like yours, which explains not just why Walsh and your reconstruction agree over the pre-1956 period, but also exposes a bias.

As we spot-check 1935 and 1952, it became clear that a low Russian Arctic extent does not automatically mean a low Atlantic or low Canadian Arctic extent.

We need to be careful with "spatial filling" a particular month, and careful not ignoring temporal dependencies. The 1935 and 1952 examples are a case in point. You can still try to argue (as you do) that the Walsh reconstruction relies too much on misplaced Kelly fields, but it is clear from DMI's direct ice observations that the Russian Arctic was low in these years and the other side of the Arctic had high extent.

Walsh reports 9.27 M km^2 for August 1952 and 6.5 M km^2 for September. That difference is unrealistic, and it is OK to acknowledge that, even if you argue that the 9.27 number is too high.

To put it differently : Based on the observational evidence, what do YOU honestly think that the Aug 1952 ice extent was ?

Diablobanquisa

Hi Rob,

Walsh reports 9.27 M km^2 for August 1952 and 6.5 M km^2 for September. That difference is unrealistic, and it is OK to acknowledge that, even if you argue that the 9.27 number is too high.

I agree, and I acknowledge that the difference between August and September 1952 on Walsh's dataset is unrealistic . But I think it's mostly due to an overestimated August extent, caused by the use of Kelly fields.


a low Russian Arctic extent does not automatically mean a low Atlantic or low Canadian Arctic extent.

I agree. But nor does it automatically imply a high Atlantic and high Canadian Arctic extent. Simply, we can't rely on Kelly grids to determine what the extent was (because Kelly fields have been proven unreliable).

To put it differently : Based on the observational evidence, what do YOU honestly think that the Aug 1952 ice extent was ?

I don't know, we should complete the direct observations using some method (analog approach, SAT-SIE climatology...) (anyway, 7.5 - 8.0 or so?)

We have a well defined sea ice edge for the whole Siberian Sector (AARI), for Barents and Svalbard (ACSYS) and for Greenland Sea (ACSYS and DMI).
(https://diablobanquisa.files.wordpress.com/2016/03/screen-shot-2016-03-13-at-12-25-42-am_zpshfyb9zauoriginal.png )
(DMI also provides a few direct observations for Baffin and CAA, but they are meaningless in order to derive a overall sea ice edge for those two regions.)

Based on the well defined sea ice edge on Greenland, Barents, Kara, Laptev, East Siberian and western Chukchi what do you think that the Aug 1952 ice extent was?

What result do you expect to get when applying your analog algorithm for August 1952? If Walsh had used his analog approach instead of Kelly fields, what do you think it would had been his result for August 1952?


Walsh's number for September 1952 is about 6.5. They got it by using the available direct observations (AARI) + analog approach.
We got almost the same number by using the available direct observations (AARI) + a climatology based on SAT-SIE correlation.

I think that 6.5 is a good estimate. The observational (direct observations) evidence from August 1952 only points out to a larger extent than September Walsh and ours at Greenland Sea.

I've always acknowledged that our approach during 1935-1952 supresses year to year variability at the American Sector, so some years could be higher and other years lower than expressed in our time series. Could September 1952 extent be larger than 6.5? Yes, it could. But, honestly, I don't think it was larger than 7.0.

Cheers


Rob Dekker

Thanks Diablo,

Couple of comments:
(1) We (actually you :o) have found that Walsh appears to have misplaced the Kelly fields by one month. In the documentation for SIBT1850, Walsh states this about potential errors :

Errors that may have occurred in the
processing to create SIBT1850 are believed to be anomalies with little effect on Arctic-wide concentration values, but a thorough assessment cannot be made at this time. Those who wish to subset the data for particular months are encouraged to view the individual months

and
These errors in concentration and source numbers will be corrected in a future version of the data set.

So he does not actually ask for feedback for correction in future versions, but I think we should tell him about these Kelly-field issues we found. I will contact him if you don't.

(2) Kelly fields run through 1956, so the mistakes (a high-bias due to the one-month displacement) should be resolved after that.

Yet when I look at years 1957 and onward, the extent is still high.
For example, here is Aug 1957 :

Extent clocks in at 9.28, almost identical to 1952. This time Kelly fields are replaced by source 5 (Walsh and Johnson) and in some places source 3 (NAVO yearbooks) besides the sources ACSYS and AARI that were used also before 1957.

The point I am making is that if the Kelly fields were wrong by a month, all data sets before 1956 should be adjusted downward.

For 1952, you suggest that instead of 9.27 is more likely to be 7.5 - 8.0, suggesting that the Kelly-field-problem is more than a million km^2.

But if we adjust these Aug numbers from 1956 and earlier downward by 1 million km^2, then there will be a 'step-up' in Aug for 1957 and later.

Unless the fields by Walsh and Johnson are also misplaced by a month too. Is that what you are suggesting ?

3) I am sorry, but I am not ready with my spatial-filling method yet. I'm still working on the algorithm, which should be simple and clean, and should produce "analogs" at the regional level and a method to merge regional analogs into a 2D pan-Arctic "best match" with uncertainty margin. And the method should be (easily) extendable to temporal constraint merging.

Diablobanquisa

Thanks, Rob.

Regarding your first comment: when I have some time, I will contact them. Anyway, you could also let them know the issue, I'm sure you'll do it with a much better and understandable english than mine.


Regarding your second comment: even though Walsh still uses Kelly fields after 1953, their weight is much more reduced than before.
From 1953 onward, Walsh incorporates source 5 and source 3. Besides, from 1953 onward DMI's direct observations at the American sector have a much wider coverage than before.
So, from 1953 onward Walsh use direct observations for most of the American sector, whereas before 1953 the American sector relied mostly on Kelly fields.
So, if there was a 'step-up', it would be mostly at the 1952/1953 boundary.


Anyway, I don't think even during 1935-1952 the bias derived from Kelly fields is systematic. The bias is different for each year, depending on:

-Kelly field itself for each year. Although most years during that period Kelly fields are quite similar, they aren't identical every year. (and, another point: which are the Kelly fields that Walsh uses during 1940-45 when DMI maps aren't available? Where do these fields come from?)

-The availability of direct observations each year: the wider coverage from direct observations, the lesser use of Kelly fields (and the lesser their effect on extent numbers).

-The actual sea ice extent, whatever it was for each year and region.

So, the bias is different each year. And, whitout and alternative methodology to infill the grid cells derived from Kelly fields, we can't even estimate an average value for that bias. So, we can't simply sustract a given value during 1935-1952 in order to get 'unbiased' data.


Regarding your third comment: don't worry, and keep it up! I think what you are trying to do is not easy at all, so I perfectly understand that you should take your time before presenting even any preliminary result. Don't hurry up!

Cheers

Diablobanquisa

I've been checking it, and during August 1953-1956 Walsh only uses Kelly fields at Baltic Sea and Sea of Okhotsk. So, from 1953 onward, they are irrelevant.


So, if there was a 'step-up', it would be mostly at the 1952/1953 boundary.

I meant a 'step-down' on the original Walsh's August numbers.

Diablobanquisa

Changing the focus, I think that source 5 (Walsh and Johnson) has also some problems for August and September from 1953 onward.

I'd say there is a concentration bias, and I think we can track it until the period 1972-1978, when Walsh dataset overlaps with the ESMR passive microwave observations.

We (as Meier et al. 2012 did) adjusted HadISST to match satellite observations on the basis of the average difference between them during the overlapping period 1972-1978. For September, HadISST presented a high bias of +0.8-0.9 M km^2.

I have calculated the average difference between new Walsh's September values and those derived from satellite observations during the overlapping period 1972-1978. The result is that the new Walsh dataset presents a high bias of +0.5 M km^2.
So, we should sustract 0.5 to their September values during 1953-1978 in order to be consistent with those derived from passive microwave satellite observations from 1979 onward.

(Before 1953, Walsh relies on different sources with a different and unknown bias, as I discussed above)

Diablobanquisa

I've also calculated the bias for August between the new Walsh dataset and passive microwave data during 1972-1978: about 1 M km^2

Diablobanquisa

I'm sorry, I made a mistake. The right biases(1972-1978)are:

- August: +0.56 M km^2
- September: +0.32 M km^2

(So, during 1953-1978 we should sustract 0.56 to August Walsh's values and 0.32 to September ones)

(The mean change from August to September during 1953-1978 on the original new Walsh dataset is 1.0 M km^2. Curiously, when the two biases have been removed the average change during the same period clocks in at 0.77 M km^2, that is closer to the average change during the satellite era: 0.7 according to NSIDC and 0.82 according to Walsh (the difference between Walsh and NSIDC is due to the fact that Walsh uses mid-month daily values instead of monthly means and G02202 instead of G02135 or GSFC))

Diablobanquisa

Left: August 1976 according to Walsh. Right: August 1976 according to ESMR passive microwave satellite data: https://diablobanquisa.files.wordpress.com/2016/03/1976_08_w_esmr.png


Look at Greenland Sea, Baffin Bay and Davis Strait, Foxe Basin, Amundsen Gulf... most of the low concentration sea ice at these regions is not seen by passive microwave satellite data.

The same in August 1974: https://diablobanquisa.files.wordpress.com/2016/03/1974_08_w_esmr_2.png

The well established 'concentration bias' between pre satellite and passive microwave satellite data and between operational maps and passive microwave ones.

Rob Dekker

Diablo,

The Aug 1974 and Aug 1976 Walsh reconstruction depends largely on source 5 "Walsh and Johnson", and it appears clear that that data set has a "high bias". Moreover, the Sept 1974 and Sept 1976 Walsh reconstruction depends on that same source and thus likely exhibits the same "high bias".

Over the 1935-1952 period, August reconstruction relies much on the Kelly fields, which we know also has a "high" bias, due to displacement by a month.

Yet 1935-1952 September in Walsh' reconstruction largely relies on source 20 (Analog filling of spacial gaps) which exhibits a "low" bias for selected years (1935 and 1952) as we discussed, and conservatively a "no" bias for other years.

As a result, it is thus quite plausible that the "uptrend" you found between 1935 and the 1970s is simply because of the high bias of the September "Walsh and Johnson" data set after 1952 and the "low" bias of the September AARI data set guiding pan-Arctic estimates for Sept 1935-1952.

Rob Dekker

Which would also explain the rather "unrealistic" differences between Aug and Sept for the pre-1953 Walsh reconstruction.

Diablobanquisa

Rob, we 'removed' the September high bias during 1953-1958 (not in Walsh but in HadISST, that is the dataset we used). So the upward trend can't be because of that high bias (we removed it).


The right biases(1972-1978)are:
- August: +0.56 M km^2
- September: +0.32 M km^2

I'd like to clarify that those are the adjustments to match Walsh's satellite numbers. In order to match NSIDC's monthly means, the adjustments would be 0.89 and 0.52

Diablobanquisa

Which would also explain the rather "unrealistic" differences between Aug and Sept for the pre-1953 Walsh reconstruction.

The differences are certainly unrealistic, but not systematicly.
The average difference during 1935-1952 is 1.25. However, some years it is about 2.0 (1935, 1942, 1949, 1951, 1952) and other years it is below 0.4 (1940, 1941, 1944, 1945)

Werther

I have some time today, trying to follow your discussion here...
For what its’ worth, while surfing on “Kelly fields Arctic” I hit upon this book:

Breaking Ice for Arctic Oil: The Epic Voyage of the SS Manhattan through the NorthWest Passage
By Ross Coen

Whatever you may think of the purpose of this book, it provides some great insight on the state of the ice in 1969 in FI McClure Strait…

Anyway, it doesn't seem to have any relation to the Kelly you guys mention...

Diablobanquisa

I suggest that in order to test whether 'AARI data set guiding pan-Arctic estimates' causes a 'high' bias or not you could do the following (when your algorithm is ready):

1. Take August data and remove all the grid cells except those derived from source 10. Look for spatial analogs and infill the gaps.
2. Remove all the grid cells except those derived from sources 13, 10 and 9. Look for spatial analogs and fill the gaps.
3. Compare both results.

(I'd suggest you could also test your algorithm during the satellite era, taking one year, randomly removing data, applying your algorithm, and testing if the result matches the actual overall extent or not).

(I also think that some Septembers during 1935-1952 Walsh is using 'infilling of temporal gaps' as well, so he could be relying on Kelly fields from the previous month (in fact, from two months earlier, due to the 'July-August' mistake) This could cause a 'high' bias at those Septembers)

Diablobanquisa

I suggest that in order to test whether 'AARI data set guiding pan-Arctic estimates' causes a 'high' bias or not

I meant 'low bias' instead of 'high bias'.


Diablobanquisa

Rob, we 'removed' the September high bias during 1953-1958

I meant 1953-1978

Diablobanquisa

Thanks Werther, some time ago I was reading that book. Anyway, as you have stated, it doesn't have any relation with 'our' Kelly.

Diablobanquisa

In summary, I think that both August and September have a high bias during 1953-1978 on the Walsh dataset. This bias is due to the 'concentration issue' and it can be quantified by calculating the average difference with satellite observations during the overlapping period 1972-1978.

I think August has a positive bias in most years during 1935-1952, due to the use of Kelly fields. However, this bias can't be easily quantified, and it seems it's not systematic during the whole 1935-1952 period. I don't think we can quantify an average bias for this period unless we create a new time series combining the available direct observations with an alternative approach to Kelly fields (your analog algorithm, our SAT-SIE correlation...).

I don't think September has a low bias during 1935-1952, because I don't think August Kelly fields are enough evidence to prove September data is low biased (the high values on August Walsh's dataset during 1935-1952 greatly rely on Kelly fields).

Some graphs:

- August and September sea ice extent on the new Walsh dataset, from 1935 until 2013: https://diablobanquisa.files.wordpress.com/2016/03/walsh_aug-sep.png


- And the differences between the two months (the bar that almost reaches 3.0 M km^2 is 1952): https://diablobanquisa.files.wordpress.com/2016/03/walsh_aug-sep_2.png


Rob Dekker

Diablo, thank you.
I have to complement you on your ability to create useful graphs so quickly. You are way ahead of me there.

Regarding that last graph, (difference between Aug and Sept over the whole series) 1935 and 1952 (the exact two years that we discussed in detail because they have such a low Sept result) stand out as having a large Aug-Sept difference. In my view, that makes the (low) Sept results for these two years suspect, especially since our detailed analysis for these years reveals that Sept results are based ONLY on AARI observational evidence, with the rest of the Arctic filled in with Walsh' spatial filling algorithm.
Moreover, if you read that algorithm in detail (chapter 8 in the , you will find that he essentially uses the 'climatology' (average) over the 1953-2000 period.

Since you also use a 'climatology' over the modern era, this explains why both your and Walsh' results are correlated for these two years specifically, and for the 1935-1952 period in general.

Are we in agreement about that ?

Rob Dekker

That said, I share you concern about the WWII period (1940-1945) when DMI did not issue sea ice graphs.
Walsh' Aug and Sept numbers over that period are flat-out weird.
It is obvious that there is something wrong there, even if you just look at his fig1 in the documentation
https://nsidc.org/data/docs/noaa/g10010-sea-ice-1850-onward/G10010_SIBT1850.pdf
Indeed it seems that Aug is exceptionally low, and virtually no ice melts out between Aug and Sept.
That can't be right, and it may make sense to study the observational sources for that period in more detail.

Diablobanquisa

Thanks, Rob.

You said: 1935 and 1952 (the exact two years that we discussed in detail because they have such a low Sept result) stand out as having a large Aug-Sept difference. In my view, that makes the (low) Sept results for these two years suspect, especially since our detailed analysis for these years reveals that Sept results are based ONLY on AARI observational evidence, with the rest of the Arctic filled in with Walsh' spatial filling algorithm.

You argue that Sep 1935 and 1952 could be low biased because the difference between Aug and Sept is too high. However, August data aren't trustworthy because they rely greatly on Kelly fields.
Even if we remove the high bias due to the use of July Kelly fields instead of August ones, August Kelly fields still could be high biased. In fact, when we have compared Kelly fields against direct observations, we have seen that they are unreliable and high biased.
In summary, the problem is the next: what do we trust more? analog spatial infilling or Kelly fields? analog spatial infilling and SAT-SIE correlation (both agree between them) or Kelly fields?
Since Kelly fields are not derived from direct observations, and they don't match direct observations from independent sources when available, I don't see any reason to trust Kelly fields, thus nor to trust the high August values on Walsh. So, until now, I don't see any reason to think that Sept values are low biased.
(your algorithm, when ready, will be an alternative approach that could help decide whether September is low biased or not, but currently I don't see evidence of that low bias).

(In fact, I don't think that during 1935-52 Walsh' August and September present a systematic bias. I'd say there are overlapping and different biases for each year and month and, as a result, the whole period is not consistent at all: August months aren't internally consistent, September months aren't internally consistent and, of course, August and September aren't consistent between them).

Diablobanquisa

September ASIE, new Walsh vs. ours: https://diablobanquisa.files.wordpress.com/2016/03/walsh_ours.png

Now, I adjust Walsh data before 1979 on the basis of the average difference between Walsh data set and satellite observations during the overlapping period 1972-1978 (although I don't think the same bias directly applies before 1953, I also adjust the 1935-1952 period, just to see what happens): https://diablobanquisa.files.wordpress.com/2016/03/walsh_adj_ours.png

Despite some discrepancies, overall both time series closely agree during the whole period. The largest difference is at 1938-1941. It would be interesting to look at these years...

Diablobanquisa

For instance, August 1939. Walsh gives for this month 9.20 M km^2. This is the map: https://diablobanquisa.files.wordpress.com/2016/03/1939_08_w.png

They are using again July (ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1939_07.jpg ) instead of August (ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1939_07.jpg ) : look at Baffin.

At Barents, they are also using July Kelly field (Svalbard is ice free because they use ACSYS instead of Kelly field. Sources (light blue is ACSYS): https://diablobanquisa.files.wordpress.com/2016/03/1939_08_w_source.png )

Looking carefully, Chukchi and the area from Mackenzie delta to Coronation Gulf are also derived from July data.

So, big errors and high biases derived from the misplacement of Kelly fields (and, even though they were using the right one, we have seen that Kelly fields themselves are unreliable and non consistent, and likely high biased).

September 1939: Walsh gives 8.46. Walsh' map: https://diablobanquisa.files.wordpress.com/2016/03/1939_09_w.png

- At Baffin Bay, the ice extent is much larger than derived from August DMI map.
- At Mackenzie-Coronation Gulf, the sea ice extent is larger than on August DMI map.
- At Greenland Sea and Svalbard, sea ice extent is much larger than on August DMI and ACSYS (ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02169/ice_edge_positions/browse/aug/iep19390812.jpg ) maps.
- At Kara Sea, ice extent is larger than on AARI August maps (ftp://sidads.colorado.edu/pub/DATASETS/NOAA/G02176/pngs/1939/w19390821_tc.v0.png ).
- Overall, all the light blue grid cells look good candidates for the concentration bias issue.

In fact, although seaice_source tells us that Walsh has used source 20, it looks more like source 21. So, instead of an spatial infilling from AARI Sept direct observations, I think they have used a temporal infilling, relying on the previous month, thus relying on Kelly fields (indeed, on July Kelly field!).

So, both August and September 1939 look high biased on the new Walsh dataset.

Diablobanquisa

They are using again July (ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1939_07.jpg ) instead of August (ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1939_07.jpg ) : look at Baffin.

ftp://sidads.colorado.edu/DATASETS/NOAA/G02203/1939_08.jpg

Diablobanquisa

1940 is a different story.
August 1940 (Walsh gives 8.16): https://diablobanquisa.files.wordpress.com/2016/03/1940_08_w.png
These are the sources: https://diablobanquisa.files.wordpress.com/2016/03/1940_08_w_source.png

So, they are combining ACSYS and AARI direct observations from Greenland Sea to Bering Strait with analog infilling for the rest of the Arctic.

Now, September 1940 (Walsh gives 8.09): https://diablobanquisa.files.wordpress.com/2016/03/1940_09_w.png

According to seaice_source (https://diablobanquisa.files.wordpress.com/2016/03/1940_09_w_source.png ) they are combining AARI direct observation from Barents to Bering with analog infilling.
However, although the sea ice edge at the Russian sector is quite similar to that in August, the result of the analog infilling is very different: more ice at eastern Beaufort and eastern Chukchi. Greenland Sea also shows more ice than in August.

Both August and September are derived from direct observations + analog infilling. However, although direct observations are quite similar, the analog approach gives rather different results for the rest of the Arctic.

So, what happens in 1940?
August has more direct observations than September. The August sea ice edge is very well constrained by ACSYS and AARI direct observations from Greenland Sea to Bering Strait. The infilling of Baffin-CAA-Beaufort-eastern Chukchi looks more realistic than when Kelly fields are involved.
In September, direct observations from ACSYS aren't available. The infilling of Greenland Sea doesn't seem to match August direct observations. The infilling of CAA-Beaufort-eastern Chukchi strongly disagrees with the infilling for the previous month. Baffin and Foxe look good candidates for a 'concentration issue'.

In 1940, I don't see any bias in August, but September looks somewhat high biased.

Diablobanquisa

1941 is more or less similar to 1940.
However, in 1942 everything changes again. I'll discuss it later.

Diablobanquisa

For August 1941 Walsh gives 8.14 and 7.74 for September. It seems quite realistic, although there are some strange changes at certain regions: for instance, sea ice extent increases at Foxe Basin from August to September. Anyway, no major issue with 1941.


1942 is completely different.
For August, Walsh gives 9.38, and 7.42 for September. According to seaice_source, both months are derived from AARI direct observations + analog infilling. However, as happened in 1940, the results of the infilling are strikingly different for both months.

August 1942 map: https://diablobanquisa.files.wordpress.com/2016/03/1942_08_w.png

The wide light blue areas at Barents, Greenland or Baffin smell like 'Kelly field', or like 'concentration bias' candidates.

Those areas completely dissapears in September: https://diablobanquisa.files.wordpress.com/2016/03/1942_09_w.png

If there is a bias, I'd choose August to be high biased. Anyway, I don't understand why the analog infilling gives so much different results for each month.

Diablobanquisa

For August 1943 Walsh gives 7.60, and 7.06 for September. Both months are derived from AARI direct observations + analog infilling. However, this year the infilling for the two months looks much more consistent. No any major issue with 1943.

For August 1944 Walsh gives 7.74 and 7.45 for September. Both months are derived from AARI direct observations + analog infilling. No major issue.

For August 1945 Walsh gives 7.43 vs 7.32 for September. Both months are derived from AARI direct observations + analog infilling. Overall, infilling is consistent, although September looks slightly high biased (for instance, sea ice extent increases from August to September at Baffin and Foxe).

Diablobanquisa

In summary:

- 1939: both August and September look high biased. August because relies on the unreliable (and misplaced) Kelly field. September because the analog infilling gives a very strange result, that is not consistent with August direct observations at all.

- When direct observations + analog infilling are applied to both months (1940-1945), August and September seem to be more consistent between them than when the unreliable and misplaced Kelly fields have a great weight in August. Nevertheless, this is not true for every year:
* In 1942, the analog infilling gives a strikingly different result for each month. Apparently, there is a high bias in August.
* In 1940, the infilling gives different results as well, but the high bias seems to be in September.
* Other years (1941, 1945) September seems to be slightly high biased.


(From 1946 onwards, August relies greatly on Kelly fields (in fact, on July Kelly fields) and every August looks high biased)

Diablobanquisa

In 1942, the analog infilling gives a strikingly different result for each month. Apparently, there is a high bias in August.

In order to visualize it better:

From this (August 1942, direct observations only):
https://diablobanquisa.files.wordpress.com/2016/03/1942_08_do.png , Walsh gets 9.28 M km^2.

Whereas from this (September 1942, direct observations only): https://diablobanquisa.files.wordpress.com/2016/03/1942_09_do.png , Walsh gets 7.42 M km^2.

In 1940, the infilling gives different results as well, but the high bias seems to be in September.

In order to visualize it better:

From this (August 1940, direct observations only): https://diablobanquisa.files.wordpress.com/2016/03/1940_08_do.png , Walsh gets 8.16 M km^2.

Whereas from this (September 1940, direct observations only): https://diablobanquisa.files.wordpress.com/2016/03/1940_09_do.png , Walsh gets 8.09 M km^2

Diablobanquisa

I think it's worth looking at these animations:

August vs. September 1935 (direct observations only): https://diablobanquisa.files.wordpress.com/2016/03/1935_08_09_do.gif

August 1935: direct observations vs. 'direct observations + Kelly field': https://diablobanquisa.files.wordpress.com/2016/03/1935_08_09_dovskelly3.gif

August 1935: direct observations vs. Kelly field: https://diablobanquisa.files.wordpress.com/2016/03/1935_08_09_dovskelly2.gif


August vs. September 1952 (direct observations only):
https://diablobanquisa.files.wordpress.com/2016/03/1952_089_08_do.gif

August 1952: direct observations vs. 'direct observations + Kelly field': https://diablobanquisa.files.wordpress.com/2016/03/1952_08_dovskelly.gif

August 1952: direct observations vs. Kelly field: https://diablobanquisa.files.wordpress.com/2016/03/1952_08_dovskelly2.gif

Rob Dekker

Thank you Diablo, for giving such a detailed analysis of these WWII years.
About 1940 :

Both August and September are derived from direct observations + analog infilling. However, although direct observations are quite similar, the analog approach gives rather different results for the rest of the Arctic.

I wondered about that too.
Not really sure how Walsh filled in Aug and Sept. When I read the documentation, its should not result in such differences for the rest of the Arctic. Maybe some undocumented part of infilling ?

Also, I'd like to get your opinion about 1945. The Aug and Sept numbers seem to be almost identical (and both very low). Any idea how that came about ?

Regarding my "analog" method, I would like to choose a method that is statistically sound, and graceful towards temporal constraints.
The issue of finding a 2D variable field based on point observations is pretty well known in climate science. For example, climate scientists reconstruct a temperature field based on spot (0D) observations all the time with PC or multi-variable analysis.

But reconstructing a (2D) ice field based on spot observations with 2D known solutions is still something else, and I have not seen an equivalent statistical method that would help out there.
So I may have to resort to a "common sense" methodology based on 'spatial' and 'temporal' separation, using some weight factors...

Rob Dekker

Ah. I just noticed that you already gave an opinion about 1945 :

For August 1945 Walsh gives 7.43 vs 7.32 for September. Both months are derived from AARI direct observations + analog infilling. Overall, infilling is consistent, although September looks slightly high biased (for instance, sea ice extent increases from August to September at Baffin and Foxe).

I'm not so convinced that "September looks slightly high biased". Especially when considering the immensely steep (2 million km^2) jump to Aug 1946, I believe that Aug 1945 was significantly low biased due to Walsh's analog filling.

Diablobanquisa

considering the immensely steep (2 million km^2) jump to Aug 1946, I believe that Aug 1945 was significantly low biased due to Walsh's analog filling.

The problem is that the steep jump is not supported by direct observations. The jump is completely due to the use of Kelly field in August 1946.
I'll try to post some animations later.

Diablobanquisa

August 1945 vs. August 1946, direct observations only: https://diablobanquisa.files.wordpress.com/2016/03/1945_46_08_do.gif

August 1946, direct observations vs. 'direct observations + Kelly field': https://diablobanquisa.files.wordpress.com/2016/03/1946_08_dovskelly.gif

Rob Dekker

Diablo,
You should really show that 1945/1946 animation to Walsh.
It is obvious that he misplaced the Kelly fields by a month, and that mistake makes a significant difference for his results.

Rob Dekker

Regarding the 1953-1977 period in Walsh's new reconstruction, I think you are right, Diablo.

The "Walsh and Johnson" source which dominates this period appears to be dominated by extrapolation and multi-variate analysis.
For example, this is what Walsh and Johnson state :


Empirical orthogonal functions of longitude are used to identify the major spatial and temporal scales of ice fluctuations within the 25-year period. The dominant spatial mode is an asymmetric mode in which the North Atlantic anomaly is opposite in sign to the anomaly over the remainder of the polar cap. A tendency for ice anomalies to persist for several months is apparent in the lagged autocorrelations of the amplitudes of the dominant ice eigenvectors. The month-to-month persistence of the ice anomalies is considerably greater than the persistence of the high-latitude meteorological anomaly fields of sea level pressure, surface temperature and 700 mb height.

http://journals.ametsoc.org/doi/abs/10.1175/1520-0485(1979)009%3C0580:AAOASI%3E2.0.CO%3B2

I don't see much mentioning of sources of observation here.

Rob Dekker

The only thing I can (with my limited 'common-sense' knowledge) is that Walsh and Johnson state that the ice extent on the Atlantic side of the Arctic has a opposite sign to the remainder of the Arctic.
Which was what I was hinting at with my 1935 and 1952 analysis.

But I don't see that conclusion reflected in observational, empirical data in the Walsh and Johnson source.

Or in plain English : I don't see how Walsh and Johnson believe that their ice concentration fields that determine much of the Arctic over the 1953-1977 period are grounded in observational data.

Rob Dekker

I'm sorry, by reading the full paper itself
http://journals.ametsoc.org/doi/pdf/10.1175/1520-0485%281979%29009%3C0580%3AAAOASI%3E2.0.CO%3B2
I come to conclude that the Walsh and Johnson IS grounded in a large number of observations (listed in table 1).

However, it is unfortunate that the exact ice observations are blended into an "ice concentration" field, so we don't know exactly which ice was observed exactly where and when.

Rob Dekker

Diablo,
I developed an method (of spatial that should work in theory.

It consists of an 2D 'analog' finder that, given an observation location will select 'analogs' that fit regionally with that observation, and
a 2D 'merger' algorithm that will determine ice extent and variability in each 1/4deg pie slice where there is no observational data, based on multiple 'analogs' and the longitudinal distance of the observation points of these 'analogs'.

I now need to implement these two algorithms, and the method.
Once that is done, I should be able to :
1) replicate Walsh' findings (by using Kelly fields as an observational source), and
2) replicate your findings (by fixing the 'climatology' you used as the only 'analog', and AARI as the only observational source, and finally
3) Come up with my own findings, by turning various sources (such as Kelly fields) off, and using the 2D merger algorithm to find ice extent with uncertainty margin for any given month in the Walsh data set.

Rob Dekker

Sorry. That first sentence should read as follows :
"I developed a method of 2D spatial filling, that should work in theory".

Note : still need to extend that to a method that uses the information from temporally separated (neighboring months) ice extent observations.

Diablobanquisa

Excellent, Rob, I'm looking forward to seeing how it works.

Rob Dekker

Diablo,
While I'm struggling to build a Netcdf4 parser from source (so I don't have to copy/paste every month of every year from Panoply) I'd like to share this temperature graphs with you :
This is NCEP/NCAR for Jan/Feb since the 1950's :
http://www.esrl.noaa.gov/psd/cgi-bin/data/timeseries/timeseries.pl?ntype=1&var=Air+Temperature&level=1000&lat1=90&lat2=70&lon1=0&lon2=360&iseas=1&mon1=0&mon2=1&iarea=1&typeout=2&Submit=Create+Timeseries

and this is Mar-Sept over the same timeframe :
http://www.esrl.noaa.gov/psd/cgi-bin/data/timeseries/timeseries.pl?ntype=1&var=Air+Temperature&level=1000&lat1=90&lat2=70&lon1=0&lon2=360&iseas=1&mon1=2&mon2=8&iarea=1&typeout=2&Submit=Create+Timeseries

The 'dip' in spring-summer temperatures in the 60's and 70's that you found clearly stands out. Which would explain the high ice-extent you found over that period.

And from the first graph, it appears that the winter temperatures do NOT show that dip. They show a much more gradual, increasing temperature profile that we would expect from ever-increasing greenhouse gases.

Just wanted to share this with you, since I find it interesting, and I wonder if you know of some effect that may be able to explain the 60's and 70's "dip" in summer (but not winter) temps across the Arctic.

P-maker

Rob & Diablo

Let me have a guess: Summer cloud cover in the Nordic Seas region increased by some 3-5 % during the 1950- 1970ies – most likely due to increasing concentration of aerosols (pollution from higher chimneys + higher consumption of dirtier fossil fuels + fewer environmental restrictions). Since then, air pollution levels have generally improved in the region and cloud cover is now back at a more natural level. The collapse of the Soviet Union around 1990 also helped to improve this situation.

More clouds during the summer months will protect the ice from melting, whereas more clouds in the dark winter months will inhibit freezing. Voila!

Rob Dekker

Thanks P-Maker, I agree that aerosols (combined with clouds or not) could explain the observed difference between the summer and winter temperature series.

Meanwhile, I have the NetCDF4 software working, and I can explore all the (1850-2013) data that Walsh provided :o)

As a sanity check, I reconstructed Walsh' "extent" numbers. Here is the reconstruction for August and September over the 1935-2013 period.

Click on the image for a larger version.
This series corresponds closely with the August and September series that Walsh reported in figure 1 of the documentation, so I am in business now and can explore a lot more.

Next, I'd like to reconstruct Diablo's series based on AARI observations and Diablo's climatology.

Rob Dekker


OK.
I took a first stab at reconstructing Diablo's AARI time series.

Or more accurately, reconstructing a time series that uses only AARI observations, and fill in the rest with the 1985-2002 'climatology' that Diablo used for the 1935-1952 period.

First I set up that 1985-2002 climatology by using the Walsh gridded data over that period.
I ended up with a gridded climatology that has 6.98 M km^s ice 'extent'.

That matches very well with the 6.99 M km^2 that Diablo reported for his climatology, which gives confidence that we are using the same gridded climatology.

Next, I ran the Walsh series, and for each grid cell if there is a AARI observation, I count that ice concentration. If the grid cell does NOT have an AARI observation, I picked the concentration from the 'climatology'.

I ran the series for September, from 1935 all the way to 1978 (the last year that there were AARI observation). This is the result :

Again, click on the image for a larger version.

Note that Diablo ran this (AARI-only, with 1985-2002 climatology) from 1935-1952.
I deliberately ran it all the way through the end of AARI observations, because that way we can say something about the 'trend' in ice extent by pure (AARI) observation, without the risk of 'tainting' by 'climatology' or switch to different sources along the way.

Floor is open for comments :o)

Rob Dekker

Let me make the first two comments :

1) Based on the AARI observations over the 1935 to 1978 period, there is a very slight up-trend, but it is NOT statistically significant.
Thus, Diablo's conclusion that "However, the trends since 1935 until early 1980s are positive and statistically significant." is falsified.

2) The reconstruction of AARI-only observations with a climatology based on the 1985-2002 average results in a time series that is consistently below the (Walsh) September series. This suggests that the 'climatology' before 1978 is NOT (is too low) compared to the 1985-2002 average, and a higher extent climatology should be chosen.

Rob Dekker

Maybe a separate, more abstract comment is needed :
The question of "why does all of this matter" ?

It matters, because if past ice extent closely follows Arctic summer temperatures, then the thickness of the ice pack also follows Arctic summer temperatures.

But if it doesn't, and past ice extent was (much) less sensitive to Arctic summer temperatures, then the ice pack in the past was much thicker than present. So much thicker that it could withstand decades-long higher summer temperatures and still survive.

Which would mean that ice thinned slowly over many decades, and it would explain why only recently (past 2 decades) ice extent seems to "nose-dive", with no apparent "buffer" to recover.

Diablobanquisa

Thanks, Rob.
So, relying on 'pure' AARI (Barents to Chukchi) data, we can see a slight upward trend from 30s to 80s.
The highest values in the 60s dissapear.

(Of, course, from 1953 onwards we do have direct observations for the rest of the Arctic as well, and the 60s' dip depends on the Atlantic and the American sectors. So, the main challenge is the infilling of the rest of the Arctic before 1953. We relied on SAT-SIE correlation for this infilling. Now, let's see how your algorithm handles it)

Cheers


Diablobanquisa

Hi Rob, I posted my comment before reading your last two posts. Some comments:


Rob wrote: Thus, Diablo's conclusion that "However, the trends since 1935 until early 1980s are positive and statistically significant." is falsified.

I don't think so. With our results, the trends are statistically significant (although they are slight trends).

(I haven't performed any statistical significance test with 'your' data, but I guess you have done.)


Rob wrote: The reconstruction of AARI-only observations with a climatology based on the 1985-2002 average results in a time series that is consistently below the (Walsh) September series. This suggests that the 'climatology' before 1978 is NOT (is too low) compared to the 1985-2002 average, and a higher extent climatology should be chosen.

The climatology is obviously too low for 1953-1978. That doesn't come as a surprise. (of course, we didn't use it for that period)

However, does it mean that the climatology is also too low for 1935-1952? Should the same climatology be applied to 1953-1978 and to 1935-1952? Why?

In order to test whether the climatology is too low for 1935-1952, you will need to run your algorithm to infill the grid cells where direct observations aren't available. (and, previously, you should test whether the algorithm works properly). Let's see.

But, until now, I can't see any evidence showing that the climatology is too low for 1935-1952 as well. In fact, the climatology match rather closely AARI direct observations for those years.

And SAT-SIE correlation strongly suggests that during the 18 year period 1935-1952 the average sea ice extent should be much closer to the average extent during 1985-2002 than to the average extent of any 18 year period during 1953-1978.

(For 1935-1952 the average Apr-Sep SAT is around 0.6 deg. warmer than for 1953-1978. For 1985-2002 the average Apr-Sep SAT is around 0.6 deg. warmer than for 1953-1978.
During 1953-2014 the correlation between SAT and SIE, smoothed with a 18 year filter, is -0.97.)


Rob wrote: But if it doesn't, and past ice extent was (much) less sensitive to Arctic summer temperatures, then the ice pack in the past was much thicker than present.

Yes, if the ice was thicker the extent could be higher despite an equally warm temperature. But, do we have any reliable data of the average thickness during the first half of the XXth century? Which was the average thickness in, for instance, 1900, 1920, 1935, 1945 and 1952?

So, yes, you can argue that the sea ice could be thicker in 1935-52 than in 1985-02, and therefore the extent larger despite the same temperature, but there is no data supporting it.

Frolov IE et al.: "The Arctic Basin:
Results from the Russian Drifting Stations". Springer, 2005. ISBN: 978-3-540-24142-3 (Print) 978-3-540-37665-1 (Online) http://link.springer.com/book/10.1007%2F3-540-37665-8 : "Noticeable climatic ice thickness fluctuations were recorded in the Arctic basin. Comparison of observational data of the expedition on Fram and G. Sedov icebreaking vessels (Buynitsky, 1951) has shown that within the 44 years separating
these observations the thickness of non-deformed ice in the near-Atlantic part of the Arctic basin decreased by 20%. Correspondingly, air temperature increased (by 6.5ºC) as did the maximum temperature of the Atlantic water (by 0.55ºC). As is
known, in the late 1930s, the first warming in the Arctic in the 20th century reached its peak."

"a linear trend of ice thickness at polar stations located in the Siberian offshore seas, determined far the period from 1936 2000, averages at only +0.2 cm/year and statistically does not differ from zero. This is also proved by results of analysing long-term ice thickness variations in fast ice of the Arctic seas (Buzuev and Dubovtsev, 2002) (...) However, the calculation of a linear trend of average thickness of fast ice within 30-year periods has demonstrated a positive tendency during 1940 1973 (+0.35 cm/year) and a negative one during 1973 2000 (-0.52cm/year)"

Diablobanquisa

So, according to these earliest available observations, sea ice thickness decreased from 1890s to 1930s, increased since 1940s until 1970s, and decreased again until 1990s.
And, according to these available observations, in the 90s the ice was as thick as in the 30s.
(obviously, during the last 15 years, sea ice has thinned much more)

AbbottisGone

That sounds like a curious history of sea ice in the Arctic if you don't mind me saying so....

Diablobanquisa

Yes, it could sound curious, but it's just the history that Russian thickness observations seem to tell, as quoted above.

Neven

Maybe it's not so curious. I saw that name pop up recently on a fake skeptic blog with many quotes from a later book by Frolov called Climate Change in Eurasian Arctic Shelf Seas.

Take this for instance:

In general, although climate models are based on physics, they inevitably include a number of adjustable parameters that are fitted to past temperature changes. We are not aware of a single climate model based on fundamental physics without adjustable parameters that has been subjected to a rigorous test against actual climate data. Climate modelers appear to assume that the Earth’s climate would continue without change, were it not for greenhouse gas emissions. They do not take into account the possibility that natural climate cycles are also acting independently of effects induced by buildup of greenhouse gas concentrations. As we have shown in Chapter 4, there is evidence for cyclic variability of Arctic climates. Furthermore, there is considerable evidence for past variability of global climate as expressed in the so-called Medieval Warm Period (900-1100) and the Little Ice Age (1600-1850). These fluctuations appear to be as great as the temperature rise of the 20th century, yet, there was no contribution of greenhouse gases to these climate changes.

A major challenge in climate modeling is to understand the range of natural fluctuations, and separate these from climate changes induced by human activity (greenhouse gas emissions, land clearing, irrigation, …). The models neglect natural fluctuations because they have no means of incorporating them, and put the entire blame for climate changes since the 19th century on human activity. As a result, they appear to project an extreme view of the future that seems unlikely to be reliable.

Or this:

In seeking a relationship between solar variability and climate change, we may consider TSI and SA (Solar Activity). The connection between TSI and climate is direct; TSI represents the fundamental heat input from the Sun that drives our climate. However, although SA represents fundamental aspects of the dynamics of the Sun, its connection to the total power emitted by the Sun is not quite clear. SA includes energetic particle emission, electromagnetic emission in the UV and higher frequency ranges and magnetic fields. It is manifested in the Earth’s phenomena in the form of polar lights, magnetic storms, radio-communication blackouts, etc. A number of different indices are used to measure the level of SA, particularly sunspot indices (Wolf number, etc.), the intensity of solar wind, and various magnetic indices. Even though variations in TSI associated with changes in SA may be small, the impact on higher latitudes is significantly amplified by the interaction of charged solar wind particles with the Earth’s magnetic field. As shown in our work, evidence exists that variability of SA is connected to Arctic climate variations.

To me that doesn't sound like a 100% reliable and objective source. But I could be wrong, of course.

Either way, I think it's nearly impossible to say something about Arctic sea ice thickness in the (recent) past, unless there have been hundreds of in-situ measurements every year. Or submarines. Maybe on longer timescales there's some proxy or other, but that's it.

Diablobanquisa

Hi Neven,
Yes, you are right, it seems that I didn't choose the most objective source...

However, that fast ice thickness record seems genuine:
Polyakov et al. 2002 (http://people.iarc.uaf.edu/~igor/research/ice/pdf/ice.pdf ):
"Figure 3 shows five 65-yr-long time series (1936-2000) of fast ice measurements from the four Arctic marginal seas. The fast ice records do not show a significant trend (Fig. 3). In the Kara and Chukchi Seas trends are positive, and in the Laptev and East Siberian Seas trends are negative. In all of the seas the trends are relatively small, about 1 cm decade, close to the resolution of the measurements. These trends are not statistically significant at the 95% confidence level."

However, with data updated until 2009, Polyakov et al. (2012, http://journals.ametsoc.org/doi/abs/10.1175/BAMS-D-11-00070.1 ) state: "Using a composite record of fast-ice thickness over the last several decades from 15 locations along the Siberian coast, it was shown that added surface melting in the eastern Arctic due to atmospheric thermodynamics caused ~0.3 m of ice thickness loss"


Either way, I agree with you, Neven, I think there is simply no enough data to say something about overall Arctic sea ice thickness before 1980s or 1970s.


Diablobanquisa

Rob, in summary, I think that your latest analysys just have shown something that we already knew: that our climatology is too low for 1953-1978. However, we don't need any climatology from 1953 onwards, we just relied on direct observations. As Walsh et al. have done.

The problem is that direct observations aren't available for most of the Arctic before 1953.

We used our SAT-SIE climatology to infill the gaps during 1935-1952. Walsh et al. have used an 'analog' methodology, and their result is reasonably consistent with ours:


If we adjust Walsh's data before 1979 on the basis of the average difference between the Walsh data set and satellite observations during the overlapping period 1972-1978 we get a closer agreement:


Now, we need to see how your 'analog' algorithm works. I think that is the most productive way to check whether September extent during the 1935-1952 period could be underestimated by our work and the new Walsh dataset.

Cheers


Rob Dekker

Hi Diablo,
I hear what you are saying and thank you for re-iterating your points that your series matches with Walsh' series.

But I would beg you to also see the limitations.
For example, the 1953-1952 period, specifically for the September series, for both Walsh and your reconstruction is exclusively from AARI, and even AARI coverage of Sept 1953-1952 is not that good.
Walsh uses a 'analog' method to fill in the rest of the Arctic, which is ultimately based on a 'climatology' that he derives from later September extents, all the way through the satellite era. And you fill in the rest of the Arctic based on also a 'climatology' based on a specific 18 year period of the satellite era. None of that is based on observation any more.

So I find it premature to draw conclusions about trends of Arctic sea ice extent, based on a few spotty AARI observations in Sept over the 1935-1952 period. Especially not if you start off with a climatology that assumes a linear dependency between SIE and summer temperatures.

That is why I think we should focus more on the August record. There are more observations there, which would give more confidence that the numbers are right, and the uncertainty will be smaller. For August, the Walsh record however is tainted by these mis-placed Kelly fields, so that (reconstructing a sensible August series) is where my efforts will be going forward.

Another comment. We do HAVE these wonderful AARI observations, and they comprise a continuous record of about half the Arctic from 1953 all the way through to 1978 (start of the satellite era). And if we look at that complete record, we do NOT see a big peak in the 60's and early 70's. So possibly that peak came from the other side of the Arctic, the side that is explicitly missing in the 1935-1952 record.
Could that uptrend come from that effect ?

Rob Dekker

Regarding land-fast ice, I would expect that to grow with winter temperatures. And winter temperatures do not seem to show the dip in the 60's and 70's. The appear to increase more gradual at least since the 50's :
http://www.esrl.noaa.gov/psd/cgi-bin/data/timeseries/timeseries.pl?ntype=1&var=Air+Temperature&level=1000&lat1=90&lat2=70&lon1=0&lon2=360&iseas=1&mon1=0&mon2=1&iarea=1&typeout=2&Submit=Create+Timeseries
But because of the SQRT function between freezing degree days (FDDs) the effect my be locally noticable buy a pan-Arctic effect my hard to detect, which explains Polyakov's conclusions.

Rob Dekker

I'm sorry for the typo's in my posts.
I promise to double-check before posting from now on.

The comments to this entry are closed.