Google Guava – sweet and succulent

I have a bit of java code that handles JWT. It generates a MACVerifier and then uses that to verify a signature. Someone commented that it was taking more time than they expected. I didn’t see a ton of opportunity for optimization, but I thought I might wrap the generation of the MACVerifier in a cache.

At first I tried EHCache. EHCache is the gold standard as far as Java caching. There are sooo many options, and there is sooo much flexibility. Write through caches, read-through caches, caches with persistence that is configurable in ways you had not imagined you needed. Java Attributes to add caching to servlets or JAX-RS. EHCache has it all.

Do One Thing Well

So I figured it would be a safe choice. But after a little bit of fiddling with it, I decided EHCache was too much. To me, EHCache violates the “do one thing well” principle of design, or if you like, the Single responsibility principle (As applied to the module, if not a particular class), or, just unsatisfying documentation which is a common problem even among “successful” open source projects.

Why is there a CacheManager? What if I create a Cache and don’t register it with a CacheManager – what happens? What do I lose? Why do I want a CacheManager? Why are there names for both managers and caches? What would happen if I registered a Cache with multiple managers? What if I don’t want persistence? What if the Cache itself goes out of scope – will it be garbage collected?

I couldn’t find ready answers to these questions and the whole experience left me lacking confidence whether the cache would do the right thing for me. In the end I concluded that EHCache was more, much more than I needed, and would require more time than I wanted to invest, to get a cache. I just wanted a simple in-memory Cache in Java with TTL support (where TTL also implies time-since-last-access or time-to-idle). And what do you know! Google Guava provides that!

Guava

Goooooooooogle

At first it was unclear how to best exploit it. But a little reading showed me that Guava has a clever design that allows the cache itself to load items into it. I don’t need to write MY code to check for existence, and then create the thing, and then put it into the cache. Guava has a LoadingCache that does all this for me. I just call cache.get() and if the item is present, it is dispensed. If it is not in the cache, then the cache loads it and gives it to me. Read-Through cache loveliness. So simple and easy.

This is my code to create the cache:

And to use the cache, I just call cache.get(). Really slick. Thanks, Google!

restclient.el – sending API Requests directly from within Emacs

Hey, something new! (to me!) the restclient.el library, for emacs. I tried it. I like it. I recommend it.

What does it do? Allows you to send REST requests (really just http requests) right from emacs, interactively. And then pretty-prints the results if possible (if XML or JSON or image). It includes a simple text mode that allows you to define a set of requests and some variables that can be used in those requests. The whole thing is simple, easy, handy.
Activate it with C-c C-c

Separately, I have a library that reads .netrc files from within elisp. It’s a natural complement to restclient.el , for API endpoints that require HTTP Basic authentication. That covers lots of API endpoints, including OAuth token dispensaries that require the client_id and client_secret to be passed in as an HTTP Basic authentication header. Here’s a simple example use:

Really nice. How did I not know about this elisp library?

One problem I had when using it: The restclient.el helpfully uses a function json-pretty-print-buffer to pretty-print the buffer containing the response, if the content-type of the response is application/json.

I don’t know that function, and it wasn’t defined on my emacs. This led to a runtime error, and a json buffer that was hard for me to visually parse.

But my emacs does have the similarly named json-prettify-buffer. So I used the following gist to get the restclient to succeed in its pretty-printing efforts.

The restclient.el module is not a huge thing, but it’s nice for us emacs people. I know about Postman, and use it. I know about Paw (but don’t use it). I know and use Fiddler. I am a big fan of curl, and someitmes curlish. This is a nice additional tool for the toolbox.  Really handy.

Thanks, Jake McCrary, for writing up your experience with Emacs and restclient.el; your blog post is how I discovered it.  And thanks of course to Pavel Kurnosov, the original author of the restclient.el library. Thanks for sharing.

EDIT – I made a change in restclient.el to fix an issue that causes an extra unintended newline to be appended to the last form parameter. This issue cost me about 90 minutes of debugging my JWT verification code, bummer! My change just trims trailing newlines from the entity being sent. This will be a problem for you if you want to send an entity that ends in several newlines. Find my fixed restclient.el here .

letsencrypt and NearlyFreeSpeech

I’ve been running this site on nearlyfreespeech for some time now.

Last week I created a cert using the tools and service made available by letsencrypt.org, and then configured my NFS server to use it. It was pretty easy, but not documented. I’ll share here what I did to make it work.

I am able to SSH into the nearlyfreespeech server. I can also perform a git clone from that server to get the letsencrypt tools. But when I ran the letsencrypt-auto tool from the server, it didn’t do what I wanted it to do. This was my first time with the tool, and I’m unfamiliar with the options, so maybe it was just pilot error.

In any case, I solved it by running the tool on my Mac OSX machine and transferring the generated PEM files to the server.

  1. I ran git clone on my local workstation (Mac OSX)
  2. from there, I ran the letsencrypt tool with these options:
    ./letsencrypt-auto certonly  --manual  \
       -d www.dinochiesa.net -d dinochiesa.net \
       --email dpchiesa@hotmail.com
    
  3. follow the instructions. I needed to create endpoints on my NFS server that responded with specific values.
  4. when that completed, I had the cert and keys in PEM format. I then copied them to /home/protected/ssl on the NFS server
  5. opened a service ticket on NFS as per This FAQ
  6. a couple hours later, the NFS people had completed the SSL config for me

Maybe this will help someone else.

It’s possible that I could have used the –manual option on the NFS Server, and avoided the need to transfer files. Not sure. If anyone else has done this, I’d like to know. I will need to renew my certs every couple months.

I’m really pleased about the letsencrypt service. I hope it gets used widely.

Update, 2017 December 7: I’ve updated my certs 3 or 4 times since I made this post. Now, this is what I do:

   sudo certbot certonly  \
     --authenticator manual  \
     --domain www.dinochiesa.net \
     --domain dinochiesa.net \
     --email dpchiesa@hotmail.com \
     --rsa-key-size 4096

I’ve automated the other parts – creating the right endpoints on the NFS server, and then copying the generated certs when they’re sent. Also NFS no longer requires a service ticket; it will automatically install certs when I update them. The change takes a minute or less. Super easy.

Use PHP code to make WordPress redirect to secure site

Lots of people use the .htaccess redirect rules to force their wordpress sites to load with the secure option.

It looks like this:

But if you have a hoster that does not provide you the ability to modify the .htaccess file, that won’t work. These hosters typically set up your server behind their load balancer which means the wordpress code sometimes cannot directly infer whether HTTPS is in use. In other words, the $_SERVER[‘HTTPS’] is not correct.

It is possible to introduce code into your theme that will do what you need. This is the PHP code:

Insert that in your theme header.php file. Or maybe the functions.php file. Invoke the maybe_redirect_to_ssl_site() function in the theme header before emitting any HTML.

Mac OSX users: update openssl. Also: openssl built-in to Mac OSX is different than brew version

If you use openssl on Mac OSX to maintain certs, etc, you should keep it up-to-date.

Worth knowing, courtesy of a comment by Gordon Davisson on THIS Stackoverflow question

…the major problem isn’t the openssl command, it’s the openssl libraries (which are used by other programs) — those aren’t API compatible between versions 0.9.x and 1.0.x, so you do not want to update the system-supplied openssl libraries!

Here’s how you get the latest openssl from brew. First, make sure you have brew installed and updated (per brew.sh):

If you already have brew installed, the output of that command will tell you.

OK, at this point you have brew installed. Then, update brew and update openssl:

If your brew is somehow broken, this command will give you lame messages. To fix, you can try “brew doctor”. (Once I resorted to simply re-installing brew. I ran the “brew install” command, which said “brew is already installed”, and then told me how to uninstall. I uninstalled, then ran the brew install again.)

Be aware that /usr/bin will probably be first on your path, so if you want to use the latest openssl you will have to explicitly request it with the fully qualified path name.

And also be aware that brew does not yet have openssl v1.0.2f, which includes the fix for CVE-2016-0701.

Pre-request script for Postman, to calculate HttpSignature

If you do REST, you probably have a favorite REST client testing tool.
Mine is Postman, a Google Chrome app.

Postman has a nifty feature called Pre-request scripts, which allows you to write some Javascript code that performs a calculation and then reads and writes the “environment” object for the request. This means you can calculate … hashes or digests or signatures or anything you like.

Here’s an example of a script that calculates an HMAC-SHA256 HttpSignature, using the keyId and secret-key embedded in the environment. It also computes a digest on the message payload. Postman helpfully includes CrytoJS in the JS sandbox, so it’s easy to do HMAC and SHA.

In this particular case, the HttpSignature verification on the server requires 2 headers (date and digest) plus the ‘(request-target)’ value. The digest is a SHA-256 of the payload, which is then base64 encoded.

Anyone can start with this and modify it to do other variations.
Good luck!

Addendum

I should have mentioned this: Postman, even the latest Chrome app version, uses XmlHTTPRequest to send out requests. XHR is purposefully limited, in the specification, to restrict some headers from being set explicitly on outbound requests. The list of restricted headers includes: Origin, Date, Cookie, Via, and others. The reason for this restriction: it is desired that the user-agent be fully in control of such request headers.

My example above uses an HttpSignature that signs the Date header. This means the code must also SET the Date header to a known value; in my case I am using a value generated by the pre-request script.

postman-set-headers

The value corresponds to “now”, the current moment. But the point is the value has to be absolutely known.

This is not possible in standard Postman, because of the XHR restriction. The effect you will see is that the intended Date header silently does not get sent with the request.

This may confound you if you don’t have an ability to trace the request on the receiving (server) side. In the context of a request that uses HttpSignature, the server will throw an error saying “Missing Date header”.

But! in Postman v0.9.6 and above, it is possible to configure Postman with something the Postman people call “the Intercptor plugin”, which then allows the lifting of this restriction. The Date header gets sent, and everything works.

If you don’t want to rely on the Interceptor plugin, and you want the HttpSignature to include the date value, then you’ll have to use a differently named header to hold the date. Use X-Date or anything other than “Date”. You need to change the client as well as the server of course, to make everything hold together.

Online calculator for SHA and HMAC-SHA

Here’s a thing I built. It’s just a webpage that calculates SHA-(1,224,256,384,512) and HMAC with the same algorithms.

I was using this to help with building a system that relies on HttpSignature. Developers need some help in constructing and validating their HMACs and SHAs.

The spec formerly known as Swagger is now OpenAPI

headline

Swagger has been renamed! Three weeks ago. I didn’t realize this, and (forgive me) I’ve been continuing to use the term “swagger” when I really should have been using “OpenAPI”, in the time since.

OAI Logo

Helpfully, Marsh, an esteemed colleague of mine, has produced a slackbot to remind me to use the world “OpenAPI” every time I type the… uh… old word… in slack chats. Now, I just need that slackbot to follow me around and remind me every time I *say* the old word.

There’s a new group, the OpenAPI Initiative, whose members include IBM, Google, Apigee, Intuit, Microsoft, Paypal… these members will govern the evolution of the spec.

REST Assured (hahahaha! ya get it?) that Apigee will be building some nice innovations on top of the OpenAPI spec. Exciting things coming soon. You can already see the beginnings at apistudio.io.

Apigee_API_Studio

It’s not difficult to imagine some interesting possible paths forward, from that tooling.

And, omigosh, I just realized that I haven’t posted an article here in about 6 months! Wow I must have been busy…