Apart from Version Control API, I've also been working on another piece of code, and after a long time of fuzzing around with architecture and internals it's now slowly getting to a state where one can get excited about it. I didn't make it in time to show it off at DrupalCon DC, but now the module can be found on drupal.org: Say hello to Transformations, your favorite new uncut diamond for performing data transformations in Drupal (including, but not limited to import/export).
Experience has shown that starting with the underlying concepts always triggers the question "So what does it actually do?", so let's start with the use cases that lead to the module's inception:
Now, I hear someone arguing like, "Scope creep! When you pack all that functionality into a single module, it will get bloated, hard to use and unmaintainable." And to a certain extent, I agree with that. As far as I can see, a major reason that Import/Export API failed is that it tried to establish a single common data format that described all possible information, and moreover, every data backend (Drupal nodes, CSV, XML) was required to cope with all that information. In other words, the architecture of Import/Export API might have required too much of a monolithic approach, which lead to the maintainability problems that were at least part of the reason why the project stalled.
Does that mean that generic import/export systems are unfeasible? I don't believe so, and I hope Transformations' architecture takes enough precautions to avoid the fate of Import/Export API.
The key idea to Transformations is that we'll never be able to capture all possible data formats and structures in a single module. What we can do, though, is to decompose that data into bite-size pieces, and define operations that can process those. (For example, operations transforming a Unix timestamp into an ISO date string or into a PHP DateTime object are pretty straightforward.) Building on that, we can assemble the pieces into larger pieces of data, and we can also define operations on those. (Extract the first three dates from a list of dates? Sure, easy enough.) In essence, importing and exporting data is nothing more than decomposing data, processing it according to a given set of rules, and reassembling it in a different form.
The key idea to Transformations is that if we want a generic import/export module, it's important not to provide a solution but to provide a framework. Transformations is a framework for creating data transformation pipelines. ETL for Drupal, you might say - a braindead attempt to eliminate a whole class of import/export modules, and a potential basis for a Yahoo! Pipes clone on Drupal. Transformations provides the means for developers and users to build their own stuff.
Oh, and Transformations is incomplete, unpolished and (beware!) object-oriented. Until now, I worked to get the concepts right. Next step: building on the foundations, and extending it to enable more concrete use cases being built upon it. For now, it can be used for CSV import/export in a way that is more complicated than Node Import will ever be. But the flexibility of the framework also promises possibilities that go way beyond nodes and two-dimensional tables.
Still reading? Still interested? Then go and check it out.