Data Visualization: Managing D3.js

d3A lot of writing energy has been put into personal use of graphics packages like D3.js, learning the Javascript, how to build those beautiful charts, dealing with bugs, etc., but the bottom line advantage of such a library is what can be accomplished in a short amount of time. We are interested in D3 because what it allows us to accomplish, and if we have only ourselves to answer to we have only ourselves to consider, but lets complicate things a bit. Let's imagine a small team with a diverse skill set, but not necessarily all in coding or web design.  Some are algorithm developers, some web designers or IT specialists, some subject matter experts and excel jockey's at best, and some just know how to schmooze a customer.  How does this team leverage D3 and push their own coding envelope while containing the pressure of building an infrastructure and new product for a customer?

As can sometimes be the case, the people coding D3 have limited access to the customer, the product designers may know code or even be part of the project code in another language but they don't necessarily know D3, the subject matter experts (SME's) have software experience at an arms length, and the customer is new to all this. Add to this differences in experience at each position, and what can be accomplished quickly comes into question due to what I call "code creep".

It's easy to say that code creep is simply spaghetti code, but that is only the end result. No one sets out to write disorganized code - except for those special individuals who see job security opportunities - in cases involving powerful libraries such as D3, spaghetti code is the result of many seemingly reasonable requests and honest attempts to meet those requests, all building on each other over time, thus the "creep". Those requests are not unreasonable, they are the result of working with the customer to provide something useful, and especially something that will sell. In this case the requests are made by intelligent people, along different links in the team communication network and therefore different project knowledge, with different focuses and understanding of what's possible, attempting to imagine and direct the next steps in their product evolution, all leading towards little green bills falling into everyone's hands.

But what is the role of D3 in all this? So far we are talking about reasonable issues in the management of many different projects. Let's consider what D3 is to someone who does not code: nothing more than a cool trick and the result of particularly capable software developers. If a person can't see how the pieces fit together, if they can't architect a solution, then the only recourse is to lean on the people who are familiar with said pieces. This is almost like a poorly designed genetic algorithm, where each evolutionary step is proposed without much knowledge of the landscape and therefore not much different than a random walk.  In other words, the team begins exploring options almost blindly.

On the other hand, knowing that the pieces can fit together is not equivalent to the level of effort required to assemble those pieces. D3 can be intoxicating and compel those using it to make promises that lead to falling green bills, but promises not supported by D3.

When we see the possibilities under D3 we want to make them the standard. We want all our work to be equally impressive. We want to say 'Yes' to the proposed path of success.  Suddenly we're in the wilderness surrounded by thorny bush code and a dull machete.  D3 can lead us to extrapolate that all other efforts must have the same high standards.  Our egos can write checks our code can't cash.