The Community Data Management Project

Help maintain the IDEA Data Store:
e-mail to impact@ideaeconomics.org

We need people to spend an hour a month helping us maintain our data.

Over the last few months we have been developing an online system for storing and analysing economic data.  The resulting web site will be a major resource. Administrators can upload data from any source, and anybody can analyse, graph and/or download it. The data can be analysed using the sophisticated scripting language. Time Series Language (TSL) is a tool developed at IDEA which allows data to be fetched, manipulated and displayed in several forms, including graphs and numerical series. Graphs created by the system will be as up-to-date as the data stored in the database and that data can come from anywhere. (We currently have debt data from the BIS, GDP data from OECD, and CPI and other data from the Federal Reserve, albeit in a very limited amount.)

The project has a few purposes:

1) To support the book which Steve Keen is currently writing.  Steve has been an eager sponsor, since the underlying OLAP structure of the data base allows powerful ways of displaying and analyzing data. Steve’s book Finance and Economic Breakdown will contain web links to graphs being generated live using the most up to date data in our system. 

2) To support graphs on the IDEA web site.  Because graphs generated by the system are always up-to-date, they will not have to be regenerated each time new data becomes available.

3) To provide a centralized source of data related to the goals of IDEA so that researchers can use it to explore their own ideas.

The data stored in the system has to be gathered from other web sites by various different means.  The data is routinely available either as a downloadable spreadsheet or via a computer to computer mechanism called an API.  Our data system can accommodate either method.

In a perfect world we could write a computer program which, each month, sucks the data electronically from its source.  In the real world this runs into the problem that the source of data may change its format in some way. In that event, the automatically imported data would not be consistent with the earlier data, but unless we were watching, we would be unaware. So the periodic acquisition of data from external sources has to be monitored by a human being, who can use their intelligence to make sure nothing odd has happened since the last import.

This may seem highly technical, but it is not.  In the end, it is just adding one or two numbers to the end of a list.  As a check we actually import numbers that we already have along with the new ones, so any mismatch with the existing data is quickly visible. If there is a mismatch, you simply abort the operation and notify us. If you are able you can try to understand the problem and suggest a solution.  

Anybody with a deeper knowledge of economic data sources can also become involved in the identification of sources for inclusion on the system which is also something we need help with.  The really motivated could take over the administration of the data curation community completely.

Our hope is that people willing to do this will talk to each other and us to contribute their ideas about the system and what data should be on it and how the system can be run better or could deliver additional functionality. 

There is of course no obligation; if enthusiasm wanes, you need only withdraw. A possibility we have discussed is a forum especially for people curating data. At a minimum, we will create an email list so curators can talk over issues with us and amongst themselves.

If you are interested in finding out more about this please send an email to impact@ideaeconomics.org