Thursday, October 14, 2010

Whatever happened to Swivel

Swivel.com was a popular data visualization site that recently failed as a business. Some information is starting to trickle out about what happened, through an interview with its founders. Some choice quotes. First, expenditure:
We got a Round A funding of two million dollars. By the time I left, we had spent about three to four million dollars on the idea.
Second, income:
[How many paying users did you have in the end?] It was single digits
Ouch. Lessons learned? I'm cherry-picking my favorite here:
I think what we learned, like Roseman is saying, that the interface is not that important, that there are analysts who are really good at tools like R, SAS, etc. and prefer to continue to work in those tools to do powerful things with datasets.
Please read the article for the full context. I just wanted to highlight this point, which I think may be important for the Data Commons Project. I think it argues for letting people maintain their data with the tools they are expert with, rather than expecting them to use a new service. My personal theory is that distributed revision control systems are the way to go for collaborative data projects. Web-based services could be part of such a distributed ecosystem, but would not be its "hub".

Tuesday, October 12, 2010

FaceBook gives users access to their own data

Good news from FaceBook: users can now download everything they've ever put on the site. This is an important step towards transparency. The move is being welcomed by the DataPortability project, whose mission is to help people to use and protect the data they create on networked services - although they are careful to note that being able to download one's data is not the same as being able to control it (read Alisa Leonard).

Monday, October 11, 2010

FACT Social Justice Challenge 2010

Voting for the FACT Social Justice Challenge 2010 is now open on NetSquared. The goal of the challenge is to use web or mobile technologies to foster collaboration around social justice issues. The DCP has a proposal in there, which we invite you to vote for:
We propose building a collaborative directory of the rooted economy.

Our goal is to develop a democratically controlled, accurate, comprehensive, publicly searchable and updatable database of cooperative economic initiatives in North America. Rather than building just another website, we're experimenting with doing so in a (de-geekified) version of how programmers develop the open-source Linux code.




A vibrant movement for a cooperative and democratic economy is growing in North America. Offering innovative and effective alternatives to the “business as usual” of the distant, unaccountable corporations that dominate our economic, social and political landscape, cooperative enterprises and the organizations that support them are building an economy that truly works for people and the planet. But these efforts are often fragmented by geography, sector, and even organizational form. To succeed in changing the economic status quo, people need to find each other, help each other, work together, and be counted.

We're working on several fronts to make this happen. We're compiling a list of directories of the rooted economy (see http://datacommoners.blogspot.com and let us know about more to add, big and small). We are on track to create a Data Commons Cooperative to foster collaboration in the long term. We are developing free and open source software to show directories and make them editable, wiki-style (see http://proto.find.coop, source code at http://launchpad.net/datacommons - this directory will be used by Cooperative Maine, the California Center for Cooperative Development, and the Regional Index of Cooperation). In the course of our work, we've found that plenty of small and large groups are willing to share their data, but that right now it costs a lot more (in time) to do so than it ought to. People like to keep their data in spreadsheets or databases on their computers, and many are reluctant to move those online (e.g. to services like Google Docs). This is understandable, since it is most comfortable to edit data this way, and databases often have fields that should not be made public or stored externally. But that means sharing data can be a tedious and messy business of filtering, exporting, emailing, hand-merging, etc. Another sore spot is making updates to information: any update made in one spot filters slowly or not at all to other locations, and takes a good deal of effort to get there. We want to create a free and open source tool to speed up the process of sharing data and maintaining it up to date.

As it happens, a lot of the pieces are already in place. Programmers have created awesome tools for collaborating in comfort among themselves, which we believe could be just as useful for stream-lined knowledge sharing. What is needed now is a small, committed team who can bring “distributed revision control systems” to our communities in a useful form.

We've already developed a free and open-source program for merging tabular data, ssmerge. There aren't many such programs out there, and we believe ours out-performs them. We are integrating that with the fossil system (http://www.fossil-scm.org) for managing shared repositories. Fossil is free and open source, by the makers of Sqlite, a database used in Firefox, Skype, Apple Macs, many smart-phones, and plenty of websites. Fossil is uniquely lightweight and dependency-free, making it a pleasure to adapt to new uses. There's a first prototype of this work at http://share.find.coop (with links to documentation and download for Linux, Windows, or Macs). We believe it could be extended to become a great tool for easily sharing parts of spreadsheets and databases, with pain-free two-way flow of updates and additions between collaborators. That would really change the culture, just like similar tools have revolutionized the software world in favor of openness and distributed collaboration. We hope FACT will support us in bringing this change to life. For programmers, distributed revision control systems such as "git" and "bazaar" have proven their feasibility, and can scale from the smallest personal project to the largest (e.g. the Linux kernel). The Data Commons Project has data-sharing partners of all sizes, so we have a strong need for a similarly flexible tool. We suspect others have this need too. Our practice and commitment is to release all our code as free and open source, in ready-to-reuse form. If successful, the tools we build will have no need for central coordination, so there will be nothing to stop our work being used in any country, by any community, without needed to talk to us.

Here are some concrete examples of who we are working to benefit. This work will benefit large umbrella organizations such as NCBA (National Cooperative Business Association) and SEN (US Solidarity Economy Network). They will be able to systematically pull data (with filtering) from small, active networks and pool data with other large peer organizations with overlapping fields of interest. Just as importantly, the work will magnify the impact of data collection activities of smaller networks, such as Cooperative Maine, that focus on specific geographies, sectors, and/or organizational forms. The data-sharing tools and repository we build will assist collaboration between those networks and the overlapping umbrella organizations that represent their interests.

In general, through the proposed work and our other efforts, here's the "status quo" we're trying to improve on (quote anonymized on request):

“For [our co-op directory] we used a database that [a large co-op] had used previously and we added minimal additional data to it—I’m not sure what its origins are, however. I found that [a large sectoral umbrella group] was unable to provide a database of their members due to agreements with their members not to share data and [another large sectoral umbrella group] does not share [member organization] data because changes occur so frequently that the database would quickly become outdated. Those two entities hold a good chunk of the database info I had hoped to collect. The smaller cooperative organizations often do not have a well organized database—I got a [small co-op sector] list in a Word doc. So, those are some of the challenges I faced. That said, I think there are many in the cooperative community who would love to see a comprehensive cooperative database come to fruition.” – a co-op data aggregator

We are excited to find ways to systematically "hold a mirror" to bottom-up community driven organizations of all kinds, so that they become visible individually and in aggregate to their peers, to researchers, and to civic institutions.

Want to help? Vote here before October 15th:

CoopMetrics

CoopMetrics is a nifty service for benchmarking a cooperative against its peers, and finding ways to improve. By pooling and comparing financial data cooperatives in a particular sector can find out what works, what doesn't, and good ideas spread faster.
  • Details on the process: In summary, a co-op's accounts are mapped onto a standardized chart of accounts, quarterly trial balances are uploaded, and CoopMetrics crunches the data to provide various reports and comparisons. There are videos that give a sense of the steps involved.
  • History: CoopMetrics traces its lineage back to 1996, with the CoCoFiSt ("Common Cooperative Financial Statements") program developed by Walden Swanson and Kate Sumberg.
This is a great example of organizations gaining an advantage by pooling information, and extracting insight that would otherwise be elusive.

Hat tip: Jim Johnson

Friday, October 1, 2010

Happy October 1!


October is Co-op Month, with lots of fun Co-opy stuff going on. If all goes on schedule, by this time next year, the Data Commons Cooperative should be emerging from a warm nest of feasibility studies and market analysis to spread its wings and take flight. The cooperative is intended as a way to sustain our work long term, but we'll be excited to work with principled organizations of all kinds.

Happy October!

Thursday, September 30, 2010

UK Open Government License


Version 1.0 of the UK's Open Government License has been released, as developed by Her Majesty’s Stationery Office (good to see a stationery office that thinks beyond paperclips and printer paper!) It is similar to a Creative Commons attribution license with some added bits. Press release here - choice quote:
Commenting on the launch of the new licence, Lord McNally, Minister for The National Archives and Public Sector Information, said: 'The National Archives isn't simply a repository of our nation's history, its task is to bring information to life, make it accessible and enable its re-use. This innovative licence gives everyone the opportunity to create products and services which benefit society.'
update: there's some information from Beth Brook about why the Open Government License was developed, rather than using a Creative Commons license. In summary: they wanted a single license that covered both copyright and database rights, covering all UK jurisdictions, and which was as simple as possible (rather than having a simple summary but complex legal language).

update 2: post on data.gov.uk about the license encouraging use of the license by local authorities.

update 3: analysis from eaves.ca on how this positions the UK.


Wednesday, September 22, 2010

Rooted economy directories

The growing list of rooted economy directories in this blog's sidebar is now a package in CKAN. CKAN is an open registry of data and content packages. You can download the directory list as a CSV file.
We maintain the list using Coopy, an experiment in distributed data collection.

"Have you and a colleague ever ended up with two versions of the same spreadsheet, with a mess of independently-made changes that now need to be merged? Coopy makes that problem go away."

Coopy applies the fossil distributed version control system, normally used for software development, to the problem of data collection.

Tuesday, September 7, 2010

More funding!

The DCP has received a grant for tech development from the Cooperative Charitable Trust (founded by Bob Giel). Thanks very much!

Thursday, August 26, 2010

Funding!

We have just received word that an application submitted back in February to fund organizational and business development of the Data Commons Cooperative has been approved. We now have the means to really research what services we can provide to whom at what cost to become a sustainable democratic business. For our early adopter data sharing organizations, this is the time to get to the nitty gritty! More details -- and ways to help us figure this out -- coming soon.

Tuesday, April 27, 2010

Private organization, open data

Infochimps has a post about the pros and cons of opening data, from the perspective of a private organization (as opposed to governments, which have been getting a lot of attention recently). Infochimps is a nifty site:
We’ve been working since the start of 2008 to build the world’s most interesting data commons, and since the start of 2009 to build the world’s first data marketplace.