rwebdb – Housekeeping
Sometimes moving forward requires a step backwards.
Sometimes moving forward requires a step backwards.
On metadata as utilities for project management and as guides to the meaning of data and how that meaning changes over time.
On extending the time series back to 1987 and extolling the virtues of minutiae.
On triangulation and techniques to build metadata and confidence in the data.
An illustration of why data curation is necessary for even modestly complex data sets.
How many of the 20,000 institutions represented in the 25 years of IPEDS data appear throughout the time series? And how many appear infrequently? Answers to these questions make an initial design decision possible and provide an opportunity to illustrate the use of an SVG data visualization.
Dynamic query programs can significantly improve management of complex datasets. Here’s a second example that illustrates the benefits.
In a data warehouse, the design effort is all about ease-of-use. Forget elegant data structures; it must first and foremost be simple to use.
Data about the data (metadata) is essential for managing a data project of any complexity.
It begins … the long but absolutely essential task of living in data long enough to recognize both its warts and its beauty.