Two quick examples. Google researchers describe “a unified Web knowledge base” as the “holy grail of web information extraction” (PDF), and the company energetically applies their research to harvest additional structured data from natural language text, HTML-embedded tables, and the so-called deep-web of databases sheltered behind form-only access.

Tim Berners-Lee, whose legacy is the web, provides another example. In a video from TED, he calls for a “new reframing” that will release the “unlocked  potential” of the web through linked data. The idea is that people and computers could traverse data related to other data, from A to B to … wherever you want to go as long as the relationships, the tuple linkings, are available. Berners-Lee’s new reframing translates into a web-of-data that could be used by scientists, citizens, social reformers, businesses, entrepreneurs, and governments in innovative new ways.

I applaud these efforts even though I realize that the potential for good and for ill seems equally strong. But my focus is nothing so grand as a web-of-data. I’d be happy if I could just answer a profoundly simple question: “Please give me a list of all web tutorials on python programming.” More symbolically I want to answer questions of the form: “Give me a list of <a> about <b>.” From there, of course, the possible questions become far more interesting.

For certain my query would be trivial if a web-of-data existed. But it does not, even though progress is being made. So, in the interim I’m looking for a substitute.

My latest excursions have taken me into the world of XML, and specifically into XQuery and, to a lesser extent, into the XQuery relatives XSLT and XPath. All of these are W3C recommendations that have been implemented in both open source and proprietary products.

Why XQuery? Well, because XML is part of the fabric of the web, and because data in XML format can be queried with XQuery, and because I stumbled upon Zorba via an O’Reilly xml.com article called “Something Tells Me You Need to Pay Attention to This.” How could I resist a title like that?

I started playing with Zorba’s XQuery about two months ago. Maybe I’ve written a couple hundred queries now. Like the learning curve for SQL, it’s clear that a couple hundred is at least an order of magnitude too small to become skilled. But it’s great fun and I’m encouraged by what I see.

One example. Tony Hirst at the Open University has done some cool things recently with The Guardian’s new data API. In one of these projects, Tony used data from The Guardian’s university guide to do a mashup on student satisfaction in architecture and planning programs at various UK universities. It featured a very nice use of DabbleDB database.

Data integration is one of the strengths of XQuery, so I set about following Tony’s lead to see if I could duplicate his mashup but by using Zorba. It was very fun and I learned tons. You can see the results in this PDF.

I’m now ready to start work on another XQuery data integration, this time using the new FRED API from the Federal Reserve Bank of St. Louis. Not that I’m particularly interested in banking-related data, but FRED uses a REST web service architecture and will allow me to play more  thoroughly with Zorba’s REST capability. And the volume of the data will allow me to stress test the performance of Zorba’s XQuery.

If you are interested at all by Zorba, I’d recommend you read some of the technical documents where you can catch glimpses of longer term development objectives and experience some of the chutzpah that must exist in the development team. A recent example is XQuery in the Browser, which was presented this week at the 18th International World Wide Web Conference in Madrid. The article basically takes aim at JavaScript. As another example, check out the plans of a 3 year-old startup called 28msec and some of their technical papers. I particularly enjoyed the architecture discussion in Donald Kossmann’s slide presentation on Building Web Applications without a DBMS (PDF).

Ok, I better stop. It’s already beginning to sound like an infomercial. Hopefully some of my sincere enthusiasm comes through, however. It’s a hopeful time. And now it’s back to making a dent in that order of magnitude learning curve.

There’s nothing at the site yet. And there won’t be for a [very?] long time. So hold your horses.

Why the announcement? A couple reasons. One, I want to make it more difficult for myself to ignore the site. And, two, it’s a small step toward the ideas I wrote about here, here, here, and here.

In all honesty, I have no idea where things will go with this. I have some vague hopes of designing and developing something [a web db?] that can then be open-sourced, improved upon, and enlivened with the new ideas of others.

My belief is that everyone everywhere has the right to learn. My hope is that someday learning will be free throughout each person’s lifetime.

It is such an implausible hope that I still get embarrassed saying it, although it’s getting easier. And when I remember the inimitable question of Richard Feynman, “what do you care what other people think?” it brings a smile.

Today in ProgrammableWeb, John Musser previewed a book by Michael Ogrinz called Mashup Patterns. In the book, Ogrinz discusses 34 types of mashups arranged in 5 categories. It was the language that Ogrinz used to describe the 5 categories that got my attention. He calls these categories harvesting, enhancing, assembling, managing, and testing.

Ogrinz is talking about the web, of course. But in the world of enterprise data warehouses, there is an interesting parallel in something called ETL. ETL is short-hand for extraction, transformation, and load. Data is first extracted from one or more data source systems. It is cleaned, aggregated, or otherwise manipulated in the transformation step. And then it is loaded into the warehouse where it is available to users, business analysts, statisticians, and policy researchers.

Data warehouses act as a middle layer between the data source systems and people who use the data in their jobs. This architectural change solves many problems, allowing data from disparate systems to be combined into a single place, inviting folks to ask new business questions, and also creating an enterprise vocabulary that helps people compare apples to apples. Most importantly, warehouses provide relatively simple and easy access points to reliable data.

I’m not suggesting that data warehouses should serve as web data systems or that mashups should comprise part of the ETL. I’m really not suggesting anything at this point, but merely using the serendipity of Ogrinz’ words to think about the basic design and development issues intrinsic in any data system build-out.