Each June, July and August, the SEARCH Sea Ice Outlook (SIO) collects predictions for the mean extent of Arctic sea ice in September. These predictions come mainly from scientists but also some other people, drawing on a variety of modeling, statistical or subjective methods that each contributor describes in written documentation. The first round of SIO predictions, submitted by June 7, have just been published online. Twenty-three contributors suggest values ranging from 3.4 to 6 million km2, with a median of 4.1.In a post last month on this blog, I invited readers to submit their own predictions. By my count a total of 116 predictions were posted by the June 7 deadline, along with some interesting discussion. My original post had an ambiguous title, leading some people to offer predictions for the one-day minimum instead of the September mean, so I view this first round as an experiment. But from here on we’ll look at how that experiment came out.
Most ASI predictions were posted here in early May, within a few days after the original post. They consequently are not informed by developments including a slow start to the melt season that became evident later that month and early June. Early submissions might be a second factor that biased predictions toward low values. Figure 2 graphs the 116 individual predictions by date submitted, with a lowess regression curve confirming that the earliest predictions tend to be lower.
The day/month ambiguity and early predictions make the first iteration experimental, but in that spirit we can still compare our crowd-source results with those of the SEARCH SIO. Figure 3 visualizes both distributions as box plots, with boxes showing the median and IQR of each group. Outliers, which are predictions more than 1.5IQR beyond the first or third quartile, are marked individually in these plots. The ASI distribution is both lower and more dispersed, as I would expect. Continuing this experiment, it will be interesting to follow both distributions over the July and August iterations, and compare their evolution with the real September ice extent.
Some of the ASI predictions reflect detailed analysis, and some are admittedly just guesses. By the same token, SIO contributions vary substantially in their basis and the expertise of their authors. The SIO distinguishes some contributors as “public” (in contrast to scientists), but if we set aside the 5 public contributions to the June SIO, the median stays the same. Resistance to outliers or distribution tails is one attraction of using medians as simple summaries for these data.SIO also distinguishes between three broad types of methods: modeling, statistical or heuristic. Figure 4 breaks down the June SIO predictions by method. From this small sample it seems that statistical approaches give more pessimistic and also more consistent results. Are statistical predictions in general likely to be more realistic, or less so? That evaluation, and how our ASI crowd source predictions fare, should be topics for post-season discussion.
The next SIO deadline is early July. You are all invited to submit, as comments to this post, your best guess for the mean September extent of Arctic sea ice (NSIDC). Your numerical prediction should be in the first line of the comment, followed by at least a sentence or two explaining the basis for your prediction — whether pure intuition, elaborate calculations, whatever you’ve got. Predictions are not bound by what you did or did not guess last month. As before, general discussion is welcome too.Results from this crowd-source experiment will be research data in their own right. Individuals can submit their own predictions directly to SEARCH, of course. The analysis here will focus on collective skill and uncertainty rather than individual. I will summarize the results after each cycle, possibly forming the basis for a research paper as well as future blog posts. The whole process should be completely transparent, because the raw data — predictions submitted here as comments — remain public and accessible to anyone.