Why I’m messing around with NoSQL

Ed Anuff —  September 7, 2010 — Leave a comment

It seems like most of my friends and colleagues have heard I’ve been using Cassandra in my current project and they forward on to me every blog post or tweet where someone has something negative to say about the Apache NoSQL open source database.  To be clear, it’s debatable whether you should use Cassandra in production today, although at the recent Cassandra Summit, it was clear a lot of people were, and were having success with it, and of course, as we all know from the blog headlines, a few people are not.  But, I also think that it’s probably useful to keep in mind how many of the basic building blocks of the web started on very shaky ground and went through many iterations before getting to where they are today.  I’m not exactly talking about the Gartner Hype Curve, because it’s very hard to apply and very hard to actually determine where you are on that curve until years after the fact.  Having ridden that curve on a number of web technologies, it’s not as simple as drawing a sloppy sideways S-curve and saying “here we are”.  The reality is that there are a number of little peaks and valleys inside the overall process.

I remember in 1995 being part of a consulting project at a Silicon Valley software company, implementing an intranet for their salesforce, where at the start of the project, the top IT people at the company were scoffing at the idea of using web technologies over Lotus Notes, because it was inconceivable to use something that didn’t have store-and-forward for offline access. Six months later, the debate was whether to use Netscape Enterprise Server or *sneer* Apache, which was just “a patchy” version of the NCSA web server. In 1997, at Wired/HotWired, the idea of building server applications in Java was something to be scoffed at. We were working with Marimba at the time, building our “push media strategy“, as well as trying to roll out a chat service built on Java. Our best engineers were highly skeptical, and eventually, for a variety of reasons, both technical and business related, we discontinued most of our Java projects. There wasn’t a tech blog scene like there is today, but if it had existed, the headlines and tweets would have been “Wired abandons Java for its sites, all web apps will be in Perl”. Subsequently, our push content would go exclusively through, um, Pointcast. Oh, and, by the way, all using Sybase.

Around the same time, “crazy” people were starting to talk about XML for content syndication. I exchanged an email with Dave Winer where he said XML was too heavyweight and “BigCo”. Luckily, he changed his mind on that, to the subsequent benefit of the blogosphere. Two years later, at Epicentric, we decided to make a very risky decision to abandon a bunch of work we’d done on a Microsoft IIS ASP-based web app and throw everything at Java and JSP. The JSP specification wasn’t even close to being finalized and the only implementation of it was something called GnuJSP, that had been written by a lone developer in Europe. Our friends who were at other Internet companies, busily keeping sites in production (this was the first Internet boom), were predictably skeptical. A few years after that, J2EE was in high gear, all enterprise web apps were in JSP, and we were doing tens of millions a year in business. Now, at the time, for the most part, everything ran on top of an Oracle database. We were hearing from really good engineers and IT people (not the stodgy IT late adopters we all love to hate) about how big a piece of s*** MySql was, and that we should steer clear of it. That attitude did a 180 turnaround pretty quick, mostly because of price, not performance. We got bought by Vignette, where they were making the risky move to serving web content dynamically, direct from the database. Existing customers *hated* this idea, much too risky. Conventional wisdom said you had to generate static pages and write them to files, which could be served much faster than being dependent on a flaky database connection. Vignette never really made that transition, they were too closely associated with static publishing. At WidgetBox we used Spring and Hibernate instead of J2EE. This was well before Spring went mainstream, making SpringSource worth close to half a billion dollars and signaling the final death knell of EJB.

Now, hopefully you haven’t been bored by this trip down memory lane, or decided that I’m stuck in the past, which is quite possible. The thing is, with every one of these choices, by the time we were hitting our stride in the market, that which was risky had become mainstream. In the portal market, our main competitor, Plumtree, had stuck to building on Microsoft ASP and found that just when the market was taking off, they had to start figuring out how to move over to Java. Now, they did a pretty good job with what they had, they were always better at sales and marketing, but if we hadn’t had the leading pure Java portal server, we’d have been out of the game rather than competing hard with BEA and IBM. These decisions are still all about placing a bet, and you can lose that bet, for example, choosing Perl, TCL, or Ruby when the market goes to PHP or whatever, so I’m not saying it’s easy, and it takes a very objective but forward looking outlook, not just picking your favorite pet language. In fact, *especially* not just picking your favorite pet language, because there are some scary expense curves involved in that, which I’ll talk about some other time.

Now, and this is very important, if you’re building a consumer Internet web app, you probably should be cobbling it together with throwaway technology, because that’s not your key value, but if you’re trying to deliver some form of infrastructure, if you’re not at the raw bleeding edge when you start, then it’s almost guaranteed that you’re not going to have enough of a technology runway to still be ahead of the pack when you go to market. And that’s why I’m tinkering with Cassandra and NoSQL, and, when the people who are the most involved in keeping today’s technology up and running sneer the loudest, why I know I’m on the right track.

POSTSCRIPT: Great quote, found in O’Reilly’s “Cassandra: The Definitive Guide“, is Ray Kurzweil’s statement “an invention has to make sense in the world in which it is finished, not the world in which it is started”.

No Comments

Be the first to start the conversation.

Leave a Reply

Text formatting is available via select HTML. <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>