This comment by long-time commenter Rob Dekker was so good and elegant that I decided to squeeze it in as a follow-up guest blog to the first Problematic predictions post.
-----
In Bill's excellent overview of correlations here, he used 'area' earlier as a predictor for 'area' later (area->area), and similarly 'extent' earlier as a predictor of 'extent' later (extent->extent).
Here, Bill noted that correlation of area->area is better than extent->extent and also that when taken over the shorter timeframes (2003->2012, 2005->2012 and 2007->2012) that the correlation factor reduces significantly (and in the case of 'extent->extent' almost completely disappears.
Others have pointed out that the correlation over the longer time frames is better because it simply qualifies the long term down trend that we all know is happening. So overall, it looks like predictions based on statistics are indeed problematic.
I think the reason that the correlations fall apart is because no physical meaning has been attached to using simply 'area' or 'extent' as a predictor for later 'area' or 'extent' of sea ice.
What I tried is to come up few variables known in June, that reflect how much energy the Arctic absorbs in June, and see if a combination (formula) of these variables works better as a predictor for 'later' 'area' or 'extent'.
I choose my variables so that they reflect the albedo ('whiteness') of the Arctic, since that determines how much solar energy the Arctic will absorb for the remainder of the season. I keep it real simple, and thus choose 3 variables for albedo:
(1) Snow - Northern Hemisphere snow cover
(2) Extent - Arctic sea ice area
(3) (Extent - Area) : Effect of the area of polynia and melting ponds throughout the ice
The 'predictor' formula (how 'white' the Arctic is) can then be:
Snow + Extent - (Extent - Area)
Now, for each of these factors, we need to determine how much of the
solar radiation will cause ice melt. As a start, which has physical
measing, I choose the following weight factors:
For (3): 1.0 (assuming that ALL solar radiation onto melting ice and into polynia will cause ice to melt later in the season.
For (2): 0.5 (assuming that half of the heat absorbed in the ocean
OUTSIDE of the main pack will cause ice melt (while the other half would
cause the ocean to warm up.
For (1): 0.25 (assuming that half the heat from lack of snow cover will be blown North, and half of that will go to ice melt.
With that rough theoretical 'guess', we then get to the formula that should serve as a 'predictor' for the amount of heat that is absorbed in June due to snow/ice cover:
0.25 * Snow + 0.5 * Extent - 1.0 * (Extent - Area)
Note that this formula expressed in simple factors is:
0.25 * Snow - 0.50 * Extent + 1.0 * Area
When I use this formula as the 'predictor' I get improved correlation
numbers (especially for the shorter terms if you use only 'Area' or
'Extent' as a predictor), which suggests we are on the right track!
But what I was really interested in, is if by tweaking these weight factors (which after all were just educated guesses), if we can improve the correlation numbers even more. If it turns out that the 'optimum' correlation is way off from the weight factors I suggested above, then we know that the physical effects of 'albedo' amplification are simply not significantly visible in the later ice cover numbers.
So I used the 1995-2012 series (long enough for statistical quantity and short enough to not be affected by completely different melting regions) and tweaked the numbers until I found optimal correlation. This was the resulting formula (normalized Area weight to 1.0):
Formula1 = 0.26 * Snow - 0.59 * Extent + 1.0 * Area
This optimal weight factor choice based on observational data is almost scarily close to the very rough numbers that were based on pretty rough theoretical estimates, and we can talk more the meaning of that that later.
For now, let me show you the initial results I get for the June->September prediction. The first two lines are the results I got by reproducing Bill's June->Sept correlations and the last two lines are the correlations I get by applying my formula to predict Sept extent/area from June extent/area/snow-cover data:
1979-2012 1995-2012 2003-2012 2005-2012 2007-2012 area->area 0.93 0.90 0.84 0.74 0.85 extent->extent 0.84 0.77 0.48 0.14 0.28formula1->area 0.94 0.94 0.93 0.90 0.90 formula1->extent 0.92 0.93 0.91 0.85 0.86
Now, what is interesting is that the correlation holds up, even for the shorter timeframes. This means that Formula1 is a better predictor that simply only 'area' or 'extent' for determining the September sea ice extent from information available in June. And the interesting thing is that this formula is based on physics of ice melt, rather than on statistical extrapolation of area and extent.
Another interesting thing is the prediction of the 2013 September ave min based on this formula. NSIDC has just published the June extent and area numbers, and in absense of Rutgers' snow cover numbers I plug in 200 k km^2 extra snow cover for June, then this formula predicts 4.5 M km^2 ice extent this September, with about 250 k km^2 standard deviation [2012 was 3.61, 2007 was 4.30 and 2011 was 4.63 million km2; N.]
Sorry for the long post, but it may be that predictions are NOT as problematic as we thought, and are NOT simply guesses, as long as we use some physically relevant data.
Much more on this later, as there is a lot more interesting info to deduce now that we know that physical effects ARE recognizable in the observational data.
And in a second comment Rob writes:
Of course weather for the remainder of the season still plays a role, but based on the simple model I presented, there is virtually no chance of reaching a record below 2012 this year.
The only thing I am specifically concerned about these holes (caused by the PAC 2013) in the Central Basin, which fall in category (3) of my formula. Heat absorbed there has the highest impact, and since they are in the Arctic Basin, they may cause quite a bit of havoc (significant ice loss in areas where we normally don't see it) later in the season.
Again, it is a very interesting melting season !
This is amazing work Rob. Thank you!
From my reading of the Student's t-test, it suggests that the probability of breaking the 2012 record is less than 0.05%, which is perhaps over-stating things, but this is the best methodology for predicting September sea-ice extent/area that I have seen.
Interestingly, your methodology suggests that the chance of this season coming in below 2007 is a bit more than 20%.
One of the interesting things about some previous records is that when a year [say 2007] has set a new record, all subsequent years also break the previous record [2005]. Will this hold?
Posted by: Misfratz.wordpress.com | July 02, 2013 at 11:55
Do any intrepid mathematicians want to see if including a variable for ice thickness improves the model further? (Thinner ice should melt faster, after all).
Posted by: Paddy | July 02, 2013 at 12:05
Also, my congratulations on some excellent work :)
Posted by: Paddy | July 02, 2013 at 12:06
Well done why the 200 k addition for the snowcover? Did you use any of the weekly numbers that are now up through week 26 for snowcover?
Posted by: Glacierchange.wordpress.com | July 02, 2013 at 13:02
Excellent stuff Rob!
Regarding Paddy's point about the "physical" versus "statistical" conundrum,
Try as one might, it seems to me that it's pretty tricky to get a good handle on thickness just at the moment. Even so, surely your stats could be improved by adding thickness into the mix somewhere, somehow?
I'm certainly still intrigued to discover how your thread over on the forum about such matters eventually pans out.
Posted by: Jim Hunt | July 02, 2013 at 13:12
Hi Rob, thanks for this method, it seems a better approach. Just curious, what did this method predict for 2012?
Posted by: Fufufunknknk | July 02, 2013 at 13:22
Hey Rob,
I am a newbie. So, this may be stupid question.
Should you not use 'area' instead of using 'extent' here.
" Extent - Arctic sea ice area". Wouldn't Area be a better indicator?
Should it not be:
Snow + Area - (Extent - Area)
Posted by: Sourabh Jain | July 02, 2013 at 13:42
Neven
Totally OT.
The Webcams page is linking to the aloftcon archive for the USCGC Healy for 2012, not 2013.
The Healy doesn't seem to have done much interesting recently but she might.
[There was a reason I didn't link properly, but can't remember what it was. Fixed now, thanks; N.]
Posted by: Glenn Tamblyn | July 02, 2013 at 14:22
Great work Rob.
Rutgers snow cover for June is 6.01 well up on 2012's 4.8 (or .2 more than 4.8? that you used?).
I am not sure what numbers you used. Should it be average snow over June or last week of June or average of April, May and June?
Similarly did you use numbers for extent and area at the end of June or average for month? (Also which extent record?)
Posted by: crandles | July 02, 2013 at 15:17
Hi, truly great simple physical model, and good agreement with observations! I have a suggestion though. Why not trying with the square root of extent in your first formula instead of extent, k1*snow - k2*sqrt(extent) - 1.0 *(extent - area), and then find the best k1 and k2 ? It loses the simplicity of a linear formula, but I think the effect of sun radiation outside the pack might be better approximated as proportional to the ice-pack perimeter, not to the extent... anyway thanks!
Posted by: Account Deleted | July 02, 2013 at 15:37
Sourabh,
The first formula
"Snow + Extent - (Extent - Area)"
made me think you end up cancelling the extents and are just left with just two factors snow + area.
However with different multipliers for the three factors, the extents do not cancel precisely and the reason for this split is in the explanation of the factors:
"For (3): 1.0 (assuming that ALL solar radiation onto melting ice and into polynia will cause ice to melt later in the season.
For (2): 0.5 (assuming that half of the heat absorbed in the ocean OUTSIDE of the main pack will cause ice melt (while the other half would cause the ocean to warm up."
In a polynia, whatever direction the wind and water current, heat moves towards ice. Outside the pack, 50% of time wind and water current move heat away from the ice.
This reasoning makes for more sophisticated analysis than just using snow + area where snow + area does make more sense than snow + extent. Note also that the extent ends up as a negative factor.
Posted by: crandles | July 02, 2013 at 15:41
Nice work. It will be extremely interesting to see how the minimum plays out this year in the context of this analysis. If it's relatively low, that would suggest that the much-mooted 'regime change' idea may be correct. Otherwise, not so much.
Posted by: Kevin McKinney | July 02, 2013 at 15:52
Today's arctic ice extent number - 9.87338 - is down 385,470 km2 from yesterday. Pop?!
Posted by: Downpuppy | July 02, 2013 at 16:07
Downpuppy: Yes! The rate of decline for the past 5 days (extent) dwarf's the rate for the same 5 days from 2012. By 35% ish. I would guess that the current condition of the ice makes all correlation analyses more or less irrelevant.
Posted by: David Nemerson | July 02, 2013 at 20:36
I agree with David, is measuring a sea ice extend of 1-2 meter thick ice the same as measuring the extent of floating wett snowballs. I think we may have been fooled by the unusually cold (and cloudy?) June and may be surprised by "what comes next".
On a side track, is it is just me or is the hurricane season refusing to start this year. Might this be related to the slowing of the jet stream that causes all these wierd temperature records in Californa, Alaska and northern Canada?
Posted by: Larsboelen | July 02, 2013 at 21:02
With 2 tropical storms already, we're a month ahead of schedule in the Atlantic -
the norm is 2 by the end of July.
http://www.nhc.noaa.gov/climo/#ori
Posted by: Downpuppy | July 02, 2013 at 21:17
Downpuppy : ah, that's just me then :-)
Thanks for the link
Posted by: Larsboelen | July 02, 2013 at 21:33
NSIDC Update: http://nsidc.org/arcticseaicenews/
Posted by: wanderer | July 02, 2013 at 22:55
@ Glenn Tamblyn, 14:22
Based on the planned 2013 scientific missions, this year's route of the Healy may be of less interest to those interested in Arctic sea ice than its 2012 tracks were.
http://icefloe.net/healy-current-mission
Posted by: Magma | July 03, 2013 at 00:02
The knee in the extent line on the NSIDC graph is still looking like it is increasing:
http://nsidc.org/data/seaice_index/images/daily_images/N_stddev_timeseries.png
And it appears to be similar to the one in early June of 2012, and it may be even steeper?
Neil
Posted by: NeilBlanchard | July 03, 2013 at 00:25
I'm flattered that Neven found my comment on Bill's post interesting enough to upgrade to a separate post.
There are several inaccuracies that slipped into some of the numbers, and the correlation link with energy absorbed is also not as straightforward as I presented, not to mention numerous typos throughout the text.
So overall I though it was a pretty sloppy post.
But reading the comments now, I'm very glad I've done so.
One thing that is important to mention is that the formula (formula1) that I presented has weight factors that were tuned to the full 1995->2012 data range.
In the table I presented, you can see that the correlation factor (for area and extent prediction) is also highest for the full range, but it drops off a bit for the years in the last decade.
Since ice conditions (thickness for example, and MYI versus FYI quantities) since 1995 most certainly changed, the I played around with the weight factors to see if I could get better decided to change the weight factors to get better correlation in more recent years.
After some tuning, I found that the following formula performs very well for recent years, while correlation does not degrade too much for the full data range :
formula2 : 0.2 * snow - 1.0 * extent + 1.0 * area ;
Let me first give the correlation factors of this new formula, for predicting Sept ice area and extent :
You can see that this formula2 is a better predictor than formula1 over the last decade, and still decent for the full 3 decade record.
Note that the increased weight on (negative) extent implies that the 'holes' in the ice (the (extent - area)) factor has become more important over the past decade w.r.t. prior decades.
As a physical explanation for that I would like to cautiously suggest that the ice in the center of the pack has become thinner, so that less melting energy is needed to widen polynias in the central pack and melting ponds melt through the bottom more often, so that final Sept area and extent is affected more strongly than before.
With some closer analysis of these 'optimized' weight factors for formulas for different decades, we may even be able to determine how thick the ice must have been in the central pack, and at the edges, during different decades... But here I'm running ahead of myself.
Overall (correlation during the last decade and relation with the physics), I'm inclined to trust this formula better than formula1 for predictions for 2013.
Some people asked what this method would have predicted for 2012, so here is the outcome formula predicted for each of the years 2003->2012, and I including a prediction for Sept 2013 Area and Extent numbers (all numbers by NSIDC monthly average) :
and
So the 2013 prediction for extent is 4.38 with SD 227 k km^2.
This prediction uses the formula2 weight factors, and includes Rutger's June snow cover (which just came out today).
Now I still take this prediction with a great deal of skepticism, mostly because I do not yet have a solid framework in place to formally test and validate these formula's let alone the method itself.
But it looks promising...
Posted by: Rob Dekker | July 03, 2013 at 09:38
Rob, thanks a lot for your great and illuminating analysis. It will be interesting to compare your method(s) with the other statistical models in 2013 and the next years.
David, the current drop rate of the extent graphs is probably not sustainable. Too much of it is coming from the 'easy' regions in the periphery. By the way, Rob's analysis suggests that extent is a negative player at this time of the year (compared to area).
Neil, the quick drop rate of the extent graphs looks similar indeed to a period in early June 2012. The CT graph is also lagging behind 2012. It will be interesting to see how this evolves in the next weeks.
[comment altered just this one time. I'm not a WYSIWYG editor :-) ; N.]
Posted by: Mignonette | July 03, 2013 at 13:22
Neven, Thank you. I really appreciate that you promoted my comment to a full post. Incidentally, since the data suggest that the (extent-area) factor (the holes in the ice) seems to become more important for the final Sept minimum, could you please post your CAPIE graph on the Arctic Sea Ice Graphs page ?
I know it's not the same thing as extent-area, but it DOES give a good idea of how 'spread-out' or 'compact' the pack is at any point in time, which appears to be more important as a predictor for trouble later.
Thanks again for great coverage of this (and previous) melting seasons, and for giving us amateurs a platform to exchange thoughts and theories about what will happen next...
Posted by: Rob Dekker | July 04, 2013 at 09:25
Rob, I will start to post the CAPIE graph in the ASI updates. To manually upload it tot he ASIG is a bit too much work.
CAPIE isn't abnormally low right now, BTW, especially now that extent is dropping faster than area. But I'm not sure the holes in the interior of the ice pack get picked up all that well.
Wipneus, of course, is doing fascinating stuff over on the Forum with the 3.125 resolution data, but it might be difficult to compare to previous years.
Posted by: Neven | July 04, 2013 at 09:31