Attempts at NoSQL

In my frantic search for the holy grail of data storage I’ve now and again been tempted to dabble in the mystic arts of NoSQL. But this is a classic case of falsely believing that there’s a silver bullet to database development. And this hasn’t happened only once. I’ve actually tried building a prototype (read: did what some tutorials told me to) from MongoDb about 3 times now. I keep forgetting why I stopped last time and I kept hitting that same wall so I figured that I’d put it in writing this time.

Documents

Talking specifically about document-databases like MongoDb, the idea that I didn’t need tables seemed novel. Why do I even need to save to multiple tables? If I saved the whole object as one document in the database, I wouldn’t need to worry about speed of writes. If I needed the object I saved back to my application, I wouldn’t need to rebuild it from multiple joins. Nor would I have to worry about things like the object relational impedance mismatch.

But now that I think about it, if all I’m worried about is that I’m not saving my data as documents, then why not just save the whole document in a column on a table? That way I can deserialize that document back to an object and solve my object-relational impedance mismatch. Fortunately, modern relational databases seem to be taking this approach. POSTGRES has extensive support for JSON types with its JSON and JSONB data types. The other relational databases seem to be catching up as well as companies like Microsoft add better support for JSON strings if not JSON types to the latest version of their software. Implementing it like this allows developers to have the best of both worlds: a document-oriented approach when you need it, while keeping all the features existing relational databases provide.

And speaking of those features, there’s a critical one that NoSQL solutions can’t seem to provide.

Transactions

At every attempt I’ve made to try to prototype Mongodb (not bashing on it, just so happened that I haven’t had time to try the others yet), it fails when I want to store multiple related documents at the same time. I have to rely on eventual consistency, and that’s where the problem comes up. There are cases where eventual consistency just won’t do. Personally, I’ve even had trouble implementing eventual consistency where it really was the only option because of the complexity it adds to an application.

Simply put these document databases aren’t ACID compliant, because maybe they’re not meant to be in the first place. But there are a lot of cases in building an application where you want to be ACID compliant. When implementing DDD for instance, I may need to update multiple aggregates in the same transaction, or save domain events in one fell swoop. Failing to save one, and allowing to save another would destroy the integrity of my data. Having the option to switch between a relational approach and a document-oriented approach as is the case with dbs like POSTGRES gives the developer freedom to apply the best tool suited for the task.

Hybrid Approaches

If you feel the need to dive into NoSQL because it’s the next big thing and you think it’ll magically solve all the difficulties you’ve had with your relational database, you may want to hold onto doing that just yet. The time you’ll spend in learning new tech may be just as well spent in improving your understanding and skills in the current ones you know. That’s where I found myself. If you’re interested in giving this a shot here a few resources which might be of help:

Wat To Do: Yet another To Do app

So I’ve set out to build my own to do app. Yeah, I know what you’re thinking, “But Jonn, there’s already a thousand to do apps already, why would you want to make another one?”. Because.. Reasons. Don’t worry. I’ve asked myself the same question a thousand times, and here’s what I came up with:

First of all, I figured that I’ve never really worked on a project that’s wholly my own. The side projects I do at home are really related to the work I’ve been doing for the past 7 years. I think I did a generic Excel parser once and dynamic fields just so I could repeat the same process when I was at work. This is probably only the second time that I’m going to work on a project I would have others use.

Another reason is that I really, really wanted to learn React.js. Been hearing a lot of good things about it. I tried Angular on my last side project (http://holidayba.com), but it’s a good thing that I postponed devoting more time to it until React came around. I work with some legacy systems (read: systems I wrote 4 or 5 years ago). Back then, as a fresh graduate, given the immense responsibility of writing a Javascript framework which would be used for future projects, I wrote a lot of code I’m not proud of today. And ever since I’ve been trying to check out Javascript frameworks along with some patterns that would help me organize the chaos that was my code. But a lot of those frameworks worked best if you already started with them and required that I overhaul the whole thing, which of course we would never be able to fit into the schedule. I hear good things about React and from what I’ve used so far, the component model is a great way to organize all these files. We’ll see.

Third, the other to do apps are not my to do app. I’ve tried a few other to do apps, and kudos to those who made them. They’re really well done. I’ve been a fan, and user of Any.Do for around 4-ish years now I think. Great design. Very simple. Loved the whole thing. I’ve flirted with one or another to do app in between too. But as I’ve learned more and more of how to think of productivity, I’ve found it hard to fit those to do apps into my own personal workflow. That and the lack of self-control and discipline I have at times when I’m at home really makes those apps useless. So I’m gonna build a to do app for lazy me. And use that to do app to help me build the to do app. If that somehow works, and it works to change my workflow for the better despite my habits, then I’d have built an app for lazy people (sample size: me). If it doesn’t work out for other people, then at least it would work out for myself. I could certainly use a good productivity boost so it works for me either way.

And lastly, this gives me something to blog about. It’s so hard to keep up this blogging habit. I know, I really, really know that writing helps me better understand what I’m doing, but it’s just soooo hard to get into the habit of it. I have to write, then read, and re-read my work. So much time is spent when I could have been straight up coding all the while. But it’s a good habit and skill to develop so I’m going to hit two birds with one stone here and blog about the experience. The danger here is that if I publicize what I’m doing, I run the risk of instantly gratifying myself from the social acknowledgment of what I’m doing. This TED talk sums up what I’m talking about here. That’s happened more than once before. But on the other hand, publicizing what I’m doing should nag me enough to keep on doing it since I now have a publicly declared obligation to keep at this habit. Or I feel that way. Here’s to hoping this whole thing works.

And with that, I’d like to introduce you to Wattodo?, a to do app for people who don’t know what to do.

Brand-marketing

I don’t know how that’s gonna do it yet, but I hope to blog more about my journey writing it. I’d be thrilled if you could join me as I try to work this out.