nodejs on Google App Engine – forcing HTTPS inbound, via HSTS

How can I force my nodejs app running on Google App Engine, to always redirect to HTTPS ?

I have a pretty vanilla app that looks like this:

This thing is running in Google App Engine (GAE), and I’d like to make sure it listens only on HTTPS. There are standards like HSTS that can help. How can I use them?

This question and answer on Stackoverflow showed me the way. Basically, just add in a tiny module called yes-https. The new code looks like this:

Redeploying (no change to app.yaml) gets me the always-HTTPS behavior I want. When a client requests my service via http, it receives a 301 redirect pointing to the secure site.

HTTP/1.1 301 Moved Permanently
Date: Wed, 20 Jun 2018 16:27:56 GMT
Transfer-Encoding: chunked
X-Powered-By: Express
Location: https://foo-bar.appspot.com/
Via: 1.1 google

Nice, easy, clear.
Thanks to Justin for this handy module.

medialize/URI.js – why’d you go and get all fancy?

I have relied on URI.js from medialize for years.

I downloaded it a long time ago, and it just works. It’s handy for parsing and building URIs form within Javascript.
I happen to use nodejs often, but I also use a JavaScript engine that runs in the JVM (via Rhino or Nashorn). So I liked URI.js for its usability across those systems.

Recently I decided to download “the latest and greatest” URI.js, and what I found… did not make me jump for joy.

URI.js is no longer “just downloadable”.

Where before I could just download the raw JS file, URI.js now has a builder that allows me to select which options I wish to include. I get the concept, and it’s a nice idea, but when I de-selected every option, I got a minimized URI.js that I did not want. When I went to the source tree I found a URI.js that included all the require() statements for punycode, Second-Level Domains, and ipv6, all stuff I did not want.

*snif*

I couldn’t figure out how to get it to “just work” in nodejs without all of that, so I had to resort to manually changing the code. Basically I just removed all the require() statements for those unneeded / unwanted modules.

And it works.

It’s possible I’m missing something basic, but for sure, it got more complicated to get the simple solution. Seems like a step backward.

Upgrade your nodejs on Mac OS X to v0.12.6

Per medium.com, there are critical vulnerabilities in nodejs and iojs. People running nodejs should upgrade to v0.12.6.

On Mac OSX, this takes just a few moments. Thanks to Stackoverflow for the tip.


~/ $ node -v
v0.10.22
~/ $ sudo npm cache clean -f
Password:
npm WARN using --force I sure hope you know what you are doing.
~/ $ sudo npm install -g n
npm http GET https://registry.npmjs.org/n
npm http 200 https://registry.npmjs.org/n
npm http GET https://registry.npmjs.org/n/-/n-1.3.0.tgz
npm http 200 https://registry.npmjs.org/n/-/n-1.3.0.tgz
/usr/local/bin/n -> /usr/local/lib/node_modules/n/bin/n
n@1.3.0 /usr/local/lib/node_modules/n
~/ $ sudo n stable

install : node-v0.12.6
mkdir : /usr/local/n/versions/node/0.12.6
fetch : https://nodejs.org/dist/v0.12.6/node-v0.12.6-darwin-x64.tar.gz
installed : v0.12.6

~/ $ node -v
v0.12.6
~/ $

I don’t really hate NodeJS

I don’t really hate NodeJS. Yes, a while ago, I said I hate NodeJS. But I didn’t really mean it. I was just suffering from unrealistic expectations. I learned to Let Go of my idea that JavaScript on MacOSX ought to be as easy as JS on Windows.

Since then I have adopted NodeJS pretty strongly. I use it for all sorts of tasks, from tools that automate Apigee Edge (nod to my employer) to API load generation utilities.

I write more code using NodeJS these days, than in anything else.

Recently I had occasion to write a little utility that computes TP99 for a set of API transactions being managed by Apigee Edge.

What it does is retrieve the transaction records logged from the Edge Analytics database, sorts them, and emits a computed TP99 (and TP95, TP90, TP50) to a Carbon server, which is backing Graphite, which then serves up charts of that data. A sample is below.

Sample TP95 chart from Graphite

For this tool, I chose NOT to use NodeJS but instead relied on good-old PERL. I didn’t want asynchrony, and I did want easy file I/O, pattern matching, and sorting. Also I wanted it to be maintainable by old-school sysadmins who no-doubt have not been following the finer points of using Q’s promises in NodeJS. Perl was the obvious choice. The sort required to compute TP99 will run on 60000 records or more, and needs to occur every minute, for all transactions logged during that minute. A cron job running a perl script was perfect for this.

But, recently I wrote another tool… this one automates the provisioning of EC2 instances in AWS, then installs Java and Apigee Edge on them, and configures them into a cluster. I wouldn’t want to do that in perl or Bash. NodeJS was the right tool for that much more complicated job. And of course there are NPM libraries for AWS, and for ssh and scp. Really helpful.

of nodejs and new clothes

A provocative post by Eric Jiang, entitles “The emperor’s new clothes were built with Node.js”, regarding the undeserved praise being heaped upon NodeJS. While I think he gets his analysis all right, he is still missing the forest for the trees.

Nodejs has grown the way it has grown for the same reason that Visual Basic and PHP grew the way they did: These things work well, and help people get things done quickly and relatively easily. And maybe, people even have fun doing it. The community support has been critical in all three cases.

Sure it’d be nice to have multi-threading like Go and a single set of known and blessed libraries like C# and performance like C. But we don’t have all those things in one package, not yet anyway.

There are weaknesses in NodeJS, just as there are in PHP and in VB. Despite that, JavaScript is effective for many many people, and will continue to be so. NodeJS makes JavaScript much, much more effective with the NPM and the vast set of downloadable modules.

No religion, yo: nodejs is not always the right choice. Of course it isn’t. I like nodejs and use it… daily. but I’m learning Go and am enjoying that as well.

NodeJS documentation is idiotic

A while back I commented on a rant by someone who said that NodeJS is stupid and if you use it, so are you.
I didn’t agree.

But, NodeJS documentation is stupid. Look, for example, at the doc for the http.request() object and it’s syntactic sugar cousin, http.get(). Let’s suppose you want to do an http get from within a node program. So you look at the doc for http.get, which says: This is like http.request(), except it sets the method to GET and calls req.end() automatically.

I wonder what that callback parameter is? http.get() accepts a callback parameter. Obviously it’s a function, but WHEN Does it get called? The doc doesn’t say.

Hmm, not very much to go on! So now I have to read the entire entry on the http.request() function. The http.get() sugar doesn’t really offer much benefit if I have to read the doc for the lower-level method, now, does it? But guess what! The doc entry for http.request() ALSO does not define the semantics of the callback parameter. It’s just a callback.

I guess if I have to ask, then I don’t need to know.

NodeJS – radical advance uses JavaScript on the Server!

I have heard a couple people in the past several weeks comment on how novel or noteworthy NodeJS is, for allowing devs to employ their beloved JavaScript language on the server!! Can you believe that? How awesome is it that we can write JavaScript on the server?!

Uh, folks, I’m here to tell ya, that ain’t new or novel.

By my reckoning, the first company to support JavaScript on the server side was named Netscape, and around 1995 or 96, they released Netscape Live Server, or something like that, which had a JavaScript programming model for web servers. Brendan Eich invented JavaScript in a rush-rush job, 10 days in May, 1995, for use in the Netscape browser. But Netscape also released a server platform that they tried to monetize, and it had a JavaScript-powered extension mechanism.

Not long after that, Microsoft released the NT Option Pack. I think this was in mid-1996. The Option Pack was a free add-on for Windows NT 4.0. An “out of band release”, they called it then. The NTOP included the first edition of ASP, what we now call “ASP Classic”, which allowed developers to use VBScript or JavaScript (Microsoft called it JScript then) to dynamically generate web content. (See The ABCs of Active Server Pages from 1997, content still on MSDN!).

ASP later became part of Windows NT, and in subsequent service packs and in Windows 2000, IIS + ASP was just “in there”. Prior to the NTOP, Windows NT 4.0 included IIS, but IIS was just a static web page server and CGI server. (I think). ASP was something a little different. By the way, ASP is still supported in IIS, yea verily, after yea these many years, 17 years by my count. So… to all those poeple who claim that Microsoft doesn’t support technologies long enough, ASP is counterexample #1.

Yes, I am not making this up: today you can install Windows Server 2008 or whatever the latest server is, enable ASP, and then deploy your JavaScript ASP code and begin serving web apps with it. It just works.

Classic ASP works for any type of content – so you can serve HTML markup of course, but also XML, or CSS (dynamically generated CSS? Maybe…) and of course client-side Javascript. This would be server-side Javascript potentially dynamically generating then emitting client-side JavaScript and yes, it is possible. Dynamic images, plaintext, of course any mime type, any content. What about REST APIs? Yes, you can use Classic ASP and JavaScript running on a Windows Server to implement a REST API, which is a use-case that is very common for NodeJS these days. Classic ASP can do content negotiation, can inspect and set cookies, can do anything a web platform can do. This also just works.

In practice, “Classic ASP” code often resembles what I will call “Rubbish PHP” code, which is to say, it mixes markup with code, willynilly, there’s poor templating, poor code re-use, poor use of classes and generally things are just an unmaintainable mess. But that is by no means required by either Classic ASP or PHP. That is just an unpleasant side effect of being really easy to use, which means novice programmers use it. Both Classic ASP and PHP have that quality. It is possible to author nicely architected ASP code, and PHP code.

Nicely architected? If I may be so bold, here’s an example.

        
        
      

In any case, NodeJS is by no means the first JavaScript-on-the-server runtime environment.

One thing that makes NodeJS different is asynchrony. Classic ASP did not support asynchrony very well.

On the other hand, NodeJS claims to be “super fast” or whatever. And from my point of view, Classic ASP running JavaScript was pretty fast dating back to 2002. Even in Windows 2000 Server, the Script host compiled the JavaScript and cached the compilation unit, so the first time you ran a page you’d incur the compile cost but thereafter all other requests executed the pre-compiled cached result. Which meant much better performance. And starting with Windows Server 2003, you could use the kernel-level cache in Windows Server, and lots of other nifty features. ASP really performed quite well. The performance of ASPNET on similar workloads is measurably better, but not by orders of magnitude. I would bet that Classic ASP with JavaScript gives NodeJS a run for its money on performance, even today.

As far as I know, Classic ASP has not been enhanced to use the new Chakra JavaScript engine. Chakra was built for IE9, the v8 competitor, and is significantly faster than the older Microsoft Script Host, about twice the throughput in the workloads I tested. I suppose that Microsoft did not see a customer benefit for using Chakra under Classic ASP. Classic ASP people aren’t clamoring for more performance out of their Classic ASP apps; at this point, they just want stability.

Another thing that sets NodeJS apart now, is the active community and the sharing. the npm is a beautiful thing, and while there were many many ASP Classic developers back in the day, there was no such thing as npm and they resorted to newsgroups and forums to share code. Primitive approaches compared to what we do now, for sure.