I have to admit that I am secretly a big fan of The Lego Movie!
It is one of those great movies that is cleverly aimed at a young and older audience all at the same time. And, although I have seen it many times (for my childrens’ sake I hasten to add), I still get a chuckle from many of the characters’ utterances.
But as I watched it, I started to notice some wonderful references to the concepts of patterns, instructions and creative adaptation that connected with my inner Data Evangelist 😀
All organisations can face real challenges if striving to make their data readily available to drive operational efficiencies, and deliver innovative products and services. Although the data they can access constantly increases in breadth, depth and sophistication, the possibility of producing holistic insights from it can become ever more remote. Indeed, even to provide basic operational transparency, can prove elusive or prohibitively expensive to fulfil.
And yet surely the basics are simple – aren’t they?
Lego Bricks aka Re-usable Data Products
All we need to do is understand our data and organise it into easily consumable units of delivery that have coherency and thus maximum re-use at their core.
Lego bricks provide a powerful metaphor for a practical delivery approach.
Each basic Lego brick has its individual characteristics, width, height, depth, colour and so on, which can be thought of as its inherent properties. The nodes on each brick allow it to be connected to other bricks which represents the interoperability of the bricks. As a result of this modular design, the number of differing structures that can be constructed belies the simplicity of each.
Core Data Products
We can think of each Lego brick as representing a Data Product – that is, a re-usable data component in our Producer-Consumer data landscape. The brick’s inherent properties represent the characterisitcs of the data that the Data Product can produce.
We can describe these fundamental bricks as our Core Data Products.
In the schematic above we can see three Core Data Products. These can be combined with other Data Products to provide ‘easy to consume’ data that delivers business value.
Consumer Data Products
In the schematic below, we see the three Core Data Products combined to form a Consumer-centric Data Product – or simply Consumer Data Product.
Here the work of ‘joining’ the data from the three Core Data Products has been carried out and published as a single composite Consumer Data Product. The Consumer Data Product may also have filtering or transformations appled to the underlying Core Data Products’ data to make it more useful to its consumers’ requirements. However, it must still have maximum re-use as a primary design imperative.
Because the Core Data Products are being used as the sources for the higher order Data Product, we can see how their re-usability is maximised. In addition, this approach has consequences directly affecting the Terms and Conditions of the Consumer Data Product. Where no transformation of the underlying data has taken place (e.g. aggregation), these are inherited from the underlying Core Data Products.
In other words, if there are restrictions on the Core Data Product’s attributes, for example GDPR, any higher order Data Products using those attributes, must also pass the restrictions on to its consumers through its Producer-Consumer contract.
This inheritance model provides further powerful benefits as the Consumer Data Product also inherits all of the data behaviours from the Core Data Products in terms of their compliance to control domains such as Data Quality and Data Retention. This build once, consume many approach maximises the benefit from Data Products and yet minimises the effort and expense to construct and maintain the Data Landscape. The approach also reduces governance costs, and improves agility by orders of magnitude, compared with the ‘one-off delivery’ contagion that is prevalent.
There are definitely limits to how far we can push this metaphor, but if trying to communicate the basic Data Products’ principles and characterisitcs it can provide a powerful yet simple metaphor.
Instruction Manuals as Data Pattern Toolkits
Another strong theme of the movie is the ability to construct entire weird and wonderful new worlds using the very simple basic bricks and adopting easy to follow Instruction Manuals. We can see how these provide the blueprints for consistent construction. And to make our approach successful, we need to emulate this Instruction Manual concept.
In our data worlds we can see these as the Data Models, Standards and Guidelines that the organisation creates and governs that bring order to what it delivers to support its operations. By providing well-formed blueprints for our entire organisation, we are able to confidently federate development and hence scale and improve responsiveness to changing requirements.
To get more of a feel for a simple approach to define your organisation’s data Instruction Manuals, have a look at this blog that outlines the Data Gym’s Data Assurance Framework. This is deesigned to act as a core data blueprint providing a template for rapid and assured development of your system landscape.
In forthcoming blogs we’ll go on to examine many other data related metaphors hidden in the movie, including architecture and agile adaptation.
