Structure is Structure – The Daily WTF

Back in the heady days of the DotCom Bubble, startups were thick on the ground, and Venture Capital money was a flood- lifting startups atop a tsunami only to crash them back into the ground a short time later. Taliesyn once worked for one such startup.

Taliesyn’s manager, Irving, was an expert in AI. In the age of the DotCom Bubble, this meant Irving knew LISP. Knowing LISP was valuable here, because their core product was a database system built on LISP- specifically the Common LISP Object System, an object-oriented bolt-on for LISP.

It was an object-oriented database system, akin to the modern NoSQL databases, but its architecture left a few things to be desired. First, since disk reads and writes were expensive operations, the system avoided them. All updates to data were done in memory, and someday, at some point, when the program felt like it, the changes would be written to disk. This meant that any failures or crashes could lose potentially days of data. Worse, the data was stored in one gigantic text file, which meant corruption could easily take out the entire database.

These were legitimate problems, and due to the architecture, they were going to be challenging to resolve. That was the startup life, however- they had a minimum viable product, and just needed to pour energy into making it something worth using.

Everyone looked to Irving, the AI and LISP expert, to guide them through this.

Irving saw where the real problems lay. “Your database doesn’t support SQL,” Irving said.

“Well, sure,” Taliesyn said. “That’s our selling point.”

Irving nodded, and then, speaking slowly, as if to a particularly dense child, said, “A database needs to support SQL.”

“I mean, a relational database, sure,” Taliesyn said, “but we’re using an object oriented data model which means we don’t need to do joins or-” Taliesyn kept talking, explaining why their database didn’t support SQL, why it shouldn’t support SQL, and why SQL support was not only off the roadmap, but so far off the roadmap that it was labelled “Here there be dragons.”

Irving nodded along, and ended the conversation with a, “Sure, that makes sense.”

Everything was fine for a few weeks, until one of Taliesyn’s peers on a different team, Angela, shot him an email: “Hey, marketing is getting antsy about a SQL demo, and I’ve got half a dozen features blocked until you get me a build with that functionality. What’s the timeline like?”

Taliesyn was uncertain about what the timeline was like, since he had now clearly slipped into a parallel universe. He politely informed his peer that he had no idea what was going on, but would find out. It didn’t take a great sleuth to figure out that Irving had started appending his own features to the roadmap.

Taliesyn tracked Irving down and demanded to know what was going on.

“A customer is already using it!” Irving protested. “They wrote it themselves! So we should be able to do it easily. Frankly, it’s embarrassing to say that we can’t do something with our own tools that the customer is already doing!”

Taliesyn knew that Irving was either wrong or lying, and asked to talk to the customer. Irving was, in fact, wrong. The customer had used LISP to write an extension to their object database (another one of those selling point features), and this extension used ODBC to open a connection to a relational database. This let them move data between the two different database systems.

“Irving,” Taliesyn said, “they’re not using SQL on our database, they’re connecting to a relational database and using SQL there. SQL doesn’t make sense for our data structure! We don’t have tables, or keys, or relationships. It’s objects! You’re promising features that we can’t deliver.”

Irving was unmoved. “We are making a database. A database must have SQL capability. It’s structured query language- it’s right there in the name! Our structure should be queried by structured query language.”

Taliesyn tried going over Irving’s head, but upper management had no interest in actually solving personnel problems. Irving’s buzzword laden ideas about why SQL was required seemed to jive with what their customers wanted, and the idea of shipping a non-SQL database was getting lost in the endless quest for the next round of VC funding.

The result was a bolted on monster of broken implementations of SQL that infuriated the few customers who tried it, Irving got promoted, and Taliesyn ran as far from the trainwreck as possible before the company flamed out.