I hate Node.js

Node.js is cool, so they say. Everybody’s doing it, so I hear.

I hate it. Node.js has wasted so much of my time, I don’t think I will ever forgive it. But take that with a grain of salt. I don’t use Node.js for its intended purpose, which is server-side Javascript.

What I want is a scripting environment for automation on the local machine. In this particular case, I want to script an FTP session from my bash prompt, I want to script a directory sync. I know I can do this with bash scripting, but I already know Javascript syntax and semantics, so I’d like to use what I know. It’s FTP in this case, but in general I want to be able to script arbitrary things on the local machine.

By the way: I can do this on Windows. I can run Javascript programs from the CMD shell, to script an FTP session, automate filesystem operations, launch applications, and a bunch of other things. It’s really nice. There’s a whole catalog of COM objects that can be scripted, including obscure stuff like the fax system in Windows, or more mainstream things like IE settings (like proxies), or IIS administration. Tons of things.

Of course I could do similar things with Node.js on MacOS, too. The problem is, the Node.js model is designed for server use. Everything is asynchronous. When I retrieve a set of files from an FTP server, and I want to iterate and retrieve each file, I have to do that asynchronously. But I don’t want it to be asynchronous. Writing asynchronous code lets me get really good throughput on a server. Writing asynchronous code for my purpose just obscures the goal of the code. I want to do this:

    fileList.forEach(function(item) {
        var modTime = ftp.getModTime(item);
        if (modTime > lastUpdate) {
            ftp.retrieveFile(item);
        }
    });

But I can’t do that. No sir. No I cannot. I’m using Node.js. And because of that, I need this:


    var c = 0, L = flist.length, 
        checkIfDone = function() {
            c++;
            if (c == L) { next(); }
        };

    fileList.forEach(function(item, ix) {
        var localPath = (dir == '.') ? item : Path.join(dir, item),
            stat, localMtime = ...,
            remoteMtime = 0;

        // get modification time of the remote file
        sync.ftp.raw.mdtm(localPath, function (e, res) {
            var d, tm;
            if (e) { }
            else {
                remoteMtime = new Date(res);
                if (localMtime == 0 || (remoteMtime > localMtime)) {
                    // Retrieve the file using async streams
                    sync.ftp.getGetSocket(localPath, function(e, sock) {
                        if (e) return console.error(e);
                        var fd = fs.openSync(localPath, "w+");

                        // `sock` is a stream. attach events to it.
                        sock.on("data", function(p) {
                            fs.writeSync(fd, p, 0, p.length, null);
                        });
                        sock.on("close", function(e) {
                            if (e) return console.error(new Error("error"));
                            fs.closeSync(fd);
                            checkIfDone();
                        });

                        // The sock stream is paused. Call
                        // resume() on it to start reading.
                        sock.resume();
                    });
                }
                else {
                    checkIfDone();
                }
            }
        });
    });

Can anyone ELSE see why I’d rather not use an asynchronous-optimized, server-oriented programming environment to script my desktop?

It’s not Javascript that’s the problem here. I can grab the v8 Javascript engine and use it to run JS code. That works. The problem is that there are no JS libraries for v8. There’s no FTP library. There’s no “any” library. The only FTP libraries I’ve found are Node-compliant. To use it, I have to do Node.js things. Likewise for all the other purposes. There’s no package manager for JS libraries, except for Node.js.

I think the solution here is for me to learn to use Bash scripting and use CURL. But that’s a lame solution. It would be nice if MacOS supported Javascript for client-side use, as nicely as Windows does. Because I hate Node.js.

The way Azure should have done it – A better Synonyms Service

This is a followup from my previous post, in which I critiqued the simple Synonyms Service available on the Azure Datamarket.

To repeat, the existing URI structure for the service is like this:

GET https://api.datamarket.azure.com/Bing/Synonyms/GetSynonyms?Query=%27idiotic%27

How would I do things differently?

The hostname is just fine – there’s nothing wrong with that. So let’s focus on the URI path and the other parts.

GET /Bing/Synonyms/GetSynonyms?Query=%27idiotic%27

Here’s what I would do differently.

  1. Simplify. The URI structure should be simpler. Eliminate Bing and GetSynonyms from the URI path, as they are completely extraneous. Simplify the query parameter. Eliminate the url-encoded quotes when they are not necessary. Result: GET /Synonyms?w=improved
  2. Add some allowance for versioning. GET /v1/Synonyms?w=positive
  3. Allow the caller to specify the API Key in the URI. (Eliminate the distorted use of HTTP Basic Auth to pass this information). GET /v1/Synonyms?w=easy&key=0011EEBB4477

What this gets you, as an API provider:

  1. This approach allows users to try the API from a browser or console without registering. The service could allow 3 requests per minute, or up to 30 requests per day, for keyless access. Allowing low-cost or no-cost exploration is critical for adoption.
  2. The query is as simple as necessary and no simpler. There is no extraneous Bing or GetSynonyms or anything else. It’s very clear from the URI structure what is being requested. It’s “real” REST.

What about multi-word queries? Easy: just URL-encode the space.
GET /v1/Synonyms?w=Jennifer%20Lopez&key=0011EEBB4477

There’s no need to add in url-encoded quotes for every query, in order to satisfy the 20% case where the query involves more than one word. In fact I don’t think multi-word would even be 20%. Maybe more like 5%.

For extra credit, do a basic content negotiation that looks at the incoming Accepts header and modifies the format of the result based on that header. As an alternative, you could include a suffix in the URI path, to indicate the desired output data format, as Twitter and the other big guys do it:

GET /v1/Synonyms.xml?w=adaptive&key=0011EEBB4477

GET /v1/Synonyms.json?w=adaptive&key=0011EEBB4477

As an API provider, conforming to a “pragmatic REST” approach means you will deliver an API that is immediately familiar to developers regardless of the platform they use to submit requests. That means you have a better chance to establish a relationship with those developers, and a better chance to deepen that relationship.

That’s why it’s so important to get the basic things right.

Azure Synonyms Service – How NOT to do REST.

Recently, I looked on the Azure data market place (or whatever it’s called) to see what sort of data services are available there. I didn’t find anything super compelling. There were a few premium, for-fee services that sounded potentially interesting but nothing that I felt like spending money on before I could try things out.

As I was perusing, I found a synonyms service. Nice, but this is nothing earth-shaking. There are already a number of viable, programmable synonyms services out there. Surely Thesaurus.com has one. I think Wolfram Alpha has one. Wordnik has one. BigHugeLabs has one that I integrated with emacs. But let’s look a little closer.

Let me show you the URL structure for the “Synonyms” service available (as “Community Technical Preview”!) on Azure.


https://api.datamarket.azure.com/Bing/Synonyms/GetSynonyms?Query=%27idiotic%27

Oh, Azure Synonyms API, how do I NOT love thee? Let me count the ways…

  1. There’s no version number. What if the API gets revised? Rookie mistake.
  2. GetSynonyms? Why put a verb in the URI path, when the HTTP verb “GET” is already implied by the request? Useless redundancy. If I call GET on a URI path with the word “Synonyms” in it, then surely I am trying to get synonyms, no?
  3. Why is the word Bing in there at all?
  4. Notice that the word to get synonyms of, must be passed with the query param named “Query”. Why use Query? Why not “word” or “term” or something that vaguely corresponds to the actual thing we’re trying to do here? Why pass it as a query param at all? Why not simply as part of the URL path?
  5. Also notice that the word must be enclosed in quotes, which themselves must be URL-encoded. That seems like an awkward design.
  6. What you cannot see in that URL is the authentication required. Azure says the authentication is “HTTP Basic Auth” which means you pass a username and password pair, joined by a colon then base64 encoded, as an HTTP Header. But… there is no username and password. Bing/Azure/Microsoft gives you an API Key, not a user name. And there’s no password. So you need to double the API key then base64 encode *that*, and pretend that it’s HTTP Basic Auth.

If readers aren’t persuaded that the above are evidence of poor API design, then you might consider heading over to the API Craft discussion group on Google Groups to talk it over.

Alternatively, or in addition, spend some time reading “the REST Manifesto,” Roy Fielding’s thesis paper, specifically chapter 5 in that document. It’s about 18 printed pages, so not too big a commitment.

The problem with releasing a poorly-designed API, is that it can do long-term damage.
As soon as a real developer takes a look at your service, he will not walk, he’ll RUN away to an alternative service. If your API is a pain to use, or is poorly designed, you are guaranteed to drive developers somewhere else. And they won’t come back! They might come just to poke around, but if they see a bad service, like this Synonyms service, they will flee, never to return. They will quickly conclude that you just don’t get it, and who could blame them?

So learn from Azure’s mistakes, and learn from the success of others. Take the time to get it right.

And now a word from my sponsor: Apigee offers a Rapid API Workshop service where we can send in experts to collaborate with your team on API design principles and practice. Contact us at sales@Apigee.com for more information.