How could Arctic data be more friendly?

How might Arctic data, such as the iconic datasets followed so closely on this blog, be made easier to access and use? In connection with a new project called the Sea Ice Prediction Network (SIPN), I’d like to collect your suggestions and pass them along to Arctic researchers.

Arctic Sea Ice blog contributors and readers tend to be active data consumers, not just of published reports but of public datasets such as those visualized in the blog’s five dense pages of graphs and maps. Many of us download numerical datasets to draw our own graphs, maps or animations. It’s a good way to learn more, try out ideas, and develop new presentations. Public data are the focus of blog discussions too, reflecting a highly engaged community of data users who are mostly not Arctic scientists themselves.

Researchers who produce the original data and publish them as a service might not have had such a citizen-science community in mind when they set up their systems. The current level of public engagement and sophistication is a relatively new thing, emerging outside the usual channels of science communication. Public-data formats designed either for scientists or for less active public consumption sometimes aren’t optimal for this new kind of use.

How might we improve that? Your suggestions are welcome. Not necessarily major changes or new graphics, but changes that hard-working researchers could make easily, to give cleaner and more public-friendly access to their numerical data. Frustrations and constructive suggestions have been voiced in some of our discussions. My hope for this thread is to collect good suggestions in one place, then start conversations with the respective research teams.

Proposed by a group of Arctic scientists, the SIPN project has support from the Arctic Sciences section of the National Science Foundation (NSF), as well as the Office of Naval Research (ONR), National Oceanic and Atmospheric Administration (NOAA), National Aeronautics and Space Administration (NASA) and the Department of Energy (DOE). Project goals are to “develop a collaborative network of scientists and stakeholders to advance research on sea ice prediction and communicate sea ice knowledge and tools. SIPN builds and expands on the Sea Ice Outlook.” More public-friendly datasets are one way to improve communication with stakeholders.

Suggestions are also welcome for original graphs, maps, animations etc. based on public data, that have been produced by participants in this blog. SIPN is hosting a “town hall” for scientists at the American Geophysical Union meetings in San Francisco this week, where I'd be happy to show a few examples of such work.

Comments

[Nightvid Cole]

It might be nice to have near-real-time maps (along with gridded data) of things like heat required to fully melt sea ice /sq meter, ice albedo, and upper ocean layer sensible heat content / sq. m. , etc.

These would allow other scientists to make predictions using their own methods without the latter being too complicated, while retaining the "semi-empirical" and relatively transparent/easy-to-understand starting points. Plus, the models could be more easily cast into a form which works with a manageable parameter set.

[Twemoran]

Andreas has some interesting musings on the topic at his blog.
http://icyseas.org/2013/11/16/simple-design-intense-content/

Terry

[Mdoliner43]

Perhaps this is slightly off topic, but I believe there is something wrong with the bot at the right hand corner of the page. According to calculations provided by (excuse it) Wikipedia, the Hiroshima Bomb released 2x10 to the 13 joules. A lightning bolt contains 5x10to the 9 joules. Yet the bot claims the extra heat is equivalent to 3.6 x 10 to the 9 bombs but only 4.6 x 10 to the 8 lightning bolts.What gives?

[Tim]

Mdoliner43: I don't know if the Wikipedia numbers are used for the app, but do note that the app indicates that the heat accumulated is equivalent to 4.6 x 10⁸ MILLION lightning bolts.

[Mdoliner43]

oops. My mistake.

[Mike]

Hi Larry, I normally feel don't comment on Neven's blog because everyone else is so erudite; but on this subject I may have something to offer. As a data analyst I am sometimes asked to 'bucket' data for marketing and management analysis, and often graph the buckets by popularity. Are big ice years getting more popular or is small ice the trend? I haven't tried it but I'd guess five buckets would do the job.

[Colorado Bob]

US Navy predicts summer ice free Arctic by 2016

Is conventional modelling out of pace with speed and abruptness of global warming?

The paper is highly critical of global climate models (GCM) and even the majority of regional models, noting that "many Arctic climatic processes that are omitted from, or poorly represented in, most current-generation GCMs" which "do not account for important feedbacks among various system components." There is therefore "a great need for improved understanding and model representation of physical processes and interactions specific to polar regions that currently might not be fully accounted for or are missing in GCMs."

http://www.theguardian.com/environment/earth-insight/2013/dec/09/us-navy-arctic-sea-ice-2016-melt

[Colorado Bob]

Study: Permafrost along Alaska Highway gradually disappearing

Permafrost along the Canadian portion of the fabled Alaska Highway is disappearing, and coverage is steadily moving north, a newly published research paper reports. Testing in 1964 found permafrost at 57 percent of sites examined along an 825-mile stretch of highway running from Fort St. John, B.C., to Whitehorse, Yukon. But in 2007 and 2008, when the 55 sites in the 1964 survey were retested, scientists found that only 29 percent still had permafrost.

The study, published in the journal Environmental Research Letters and discussed about a month later at a permafrost conference at Purdue University, was conducted by researchers at the University of Ottawa and the Geological Survey of Canada.

http://www.alaskadispatch.com/article/20131208/study-permafrost-along-alaska-highway-gradually-disappearing

[Connie Quirk]

Hi, Bob--

The Guardian is rehashing Maslowski, I'm afraid. Nothing really new there (the linked review paper is from spring 2012.) FWIW, I don't think conventional wisdom among modelers puts the first ice-free summer as far away as 2100 any more; the consensus looks to be more on the order of 2030-2040.

But returning to the topic of Neven's post, I'd like to see more data available as CSVs or something similar; there are a number of products online for which typical home computers don't have appropriate software. (It would be great, for us duffers, to be able to us non-specialist software like Excel--a lot of us will never be reasonably justified in acquiring and learning R.)

I wonder if that could be a citizen scientist initiative, BTW. I know there are a number of initiatives like this one:

http://www.noaanews.noaa.gov/stories2012/20121022_oldweatherprojectlaunch.html

Maybe volunteer coders could team with agencies holding records to build programs to reformat data, which the latter could then host? I can imagine some issues with this, but maybe they wouldn't be insuperable.

[L. Hamilton]

But returning to the topic of Neven's post, I'd like to see more data available as CSVs or something similar; there are a number of products online for which typical home computers don't have appropriate software. (It would be great, for us duffers, to be able to us non-specialist software like Excel--a lot of us will never be reasonably justified in acquiring and learning R.)

Thanks, Connie, I've been thinking along csv lines too -- it should be a simple fix for most sites. Do you have particular examples in mind?

[L. Hamilton]

As a data analyst I am sometimes asked to 'bucket' data for marketing and management analysis, and often graph the buckets by popularity. Are big ice years getting more popular or is small ice the trend? I haven't tried it but I'd guess five buckets would do the job.

Mike, can you suggest an example of what this might look like?

[Mike]

Larry, I think Connie is on much the same track as me. I'll need to try and work up a graphic example, which may not be today. I'm going to follow the link that Connie supplied as well. I don't think this is about prediction as much as interpreting past events in a user friendly fashion using a tool like Excel and pivot charts. I was going to look at Neven's data sets on Google share for source data. I like the way that warming decades can be presented as a column chart. Like this one http://www.washingtonpost.com/blogs/wonkblog/wp/2013/07/09/you-cant-deny-global-warming-after-seeing-this-graph/

[Hans Gunnstaddar]

Great graph, Mike! Averages of each decade really brings home view of what is happening; warming. Like we didn't know from Arctic melting, but nice to confirm.

[wayne]

Very a propos Neven, I am looking for High Resolution SST graph for the Arctic, and I usually get so poor resolution, the Canadian Islands look like smudges. I am wondering if we live on blots or Islands :)

To be friendlier, Us folks in the Arctic need to write more. But to a certain degree of satisfaction, the Arctic was named a lot during this latest US cold spell. However , the Geography of its origin is off. Nothing unusual, we get mentioned when we should not be, then, when something really big happens up Here, its summer and Baseball is more important. http://eh2r.blogspot.ca/

[Bob Wallace]

I've got a couple of suggestions at the butt-simple end of things.

1) Standardize colors. Make 2010 the same color on every graph, regardless of who generates it.

2) Standardize globe rotation. I don't care if Greenland is top, bottom, right or left. Just put it in the same orientation on all graphs.

And it would be nice if everyone used the same boundaries for regions.

OK, put a date on each chart. Nice and clear. No reason why people should have to wonder when the last update was.

[Wipneus]

R and Python are perfectly happy with text files that can be read in spreadsheets, either csv or fixed-width columns. The latter is even generated/processed easy by FORTRAN, so "real" programmers wont be left out.

It is nice to include a line with the names of the column headers. In a program they can end as variable and field names. Please only use alpha-numeric characters, no spaces in particular.

What is a royal pain sometimes, is the comments and other meta data in the same files on lines not starting with the comment character (commonly '#'). It requires either some pattern recognition or just line counting in the reader.

Date and time should be in a standard notation, use ISO8601 and the pain is evenly distributed over the world.

And please, please, make sure day and time should be UTC, or document the time zone. Some of the sea ice extent series have noticeably offsets of a half day compared with others.

Where time is in years before present, the "present" should be noted everywhere unambiguously.

[Chris Reynolds]

Wayne,

I've got a 2423 X 2514 Jpeg, which I think is the version produced by Jakobson 2008, is that big enough?

As for data. There's stacks of it, I get the impression those interested are already using it, which just leaves those who aren't, who aren't interested.

[plus.google.com/102121405461486954917]

I like data expressed as standard deviations.

[Mike]

I've just been having a play with the global climate dashboard at http://www.climate.gov/ . Very easy to use for the lay person and with links to the data. Although it would be nice to have a csv to download. Chris's point that there is stacks of data is true, but there are issues with getting it in a format that us mere mortals with an excel spreadsheet can use. Plus.google... raises the point of SD and that may be useful as well. It would be nice to have that dashboard as a download, or a plug-in, so that it can be viewed separately or linked to.

[Mike]

Wipneus, I was just reading your comment and thought I should say how much I agree with what you are saying about data formatting, and that's from an former FORTRAN programmer!

[Chris Reynolds]

Mike,

I've produced meta data from PIOMAS gridded data. I use Excel VBA macros to process the source binary files containing PIOMAS gridded data.

You'll find regional breakdowns of PIOMAS gridded data in three different approaches here:
http://dosbat.blogspot.co.uk/2013/12/regional-piomas-volume-data.html