Loving the simple API Design Guidelines from GoCardless

See here.

I like this for several reasons:

  • I like the simplicity and clarity of the guidelines.
  • I agree with all of their guidelines; nothing feels controversial there. Such as: Use JSON, and pretty print it. Be explicit with error messages. Use plural nouns for containers. Etc.
  • I like the fact that it is open sourced for the world to see, share and fork.

ps: My employer, Apigee, is still looking to hire SEs, and other API geeks.

Uber aims to be a platform

You may think of Uber as a disruptive player in the hire-car business. They offer a service similar to taxis, but enabled with an app experience that riders really appreciate. The hailing and pay-by-phone experience is something that I personally love. I love knowing where the driver is, how long it’ll take to arrive, the ease of paying, and so on.

But now Uber is launching an API. They’re aware that disruption is not simply something they dish out to others, but that they are also subject to disruption. They cannot afford to stand still.

Uber has a key asset – a network of drivers available on a for-hire basis. By exposing that asset programmatically via an API, Uber turns itself from simply a taxi service with an app, into a people-and-goods delivery platform upon which other companies can build their businesses. Pharmacies are a key potential partner – think of Prescription delivery services for people with limited mobility. Uber can act as a personalized delivery service for any high-value retail good.

Or think of personal concierge services – what if you could build an app to request a car to any location on behalf of a VIP or a person you’re trying to impress?

Uber have announced 11 partners at the launch of their public API. Obviously the market is interested in building new businesses or expanding existing businesses in partnership with them.

The beautiful thing about APIs is that any business can expose their capabilities programmatically to transform into a “platform”. That means greater reach, more partnerships, lower cost of business development, better return on assets.

Think back: this is exactly the kind of nirvana we in the industry were envisioning, 15 years ago, when we created SOAP and all the WS-* specifications. But we went too far out on the complexity-vs-value spectrum. It turns out the WS-SecureConversation spec was not the key concept that would enable flexible business partnerships. We now realize that simplicity of design and ease of programmability are the keys to unlock the possibilities in this domain. I’m so glad that time has arrived!

What’s the basis for your platform?

Evernote’s argument for delivering a REST-less API leaves me unimpressed.

The Evernote API is notable because it is not based on REST. The defense of that decision leaves me unimpressed.

When the world is going to REST, fully open and usable APIs, why would Evernote go the other way? They ought to have a good reason. Evernote’s VP of Platform Strategy Seth Hitchings has something to say about it. According to the article on ProgrammableWeb,…

Hitchings concedes that compared to the RESTful APIs, developers have to endure a bit of a learning curve to make use of the SDKs’ core functionality; to create, read, update, search, and delete Evernote content. But then again, according to Hitchings, Evernote is a special
needs case

OK, so it’s more work for the consuming developers. It’s also more work for the company, because they have to support all the various “SDKs”, as they call them. [Evernote delivers libraries for various platforms including iOS, Android, C#, PHP, JavaScript, and more. They call these things “SDKs”, but they’re really not SDKs. An SDK is a Kit, that includes a libraries, documentation, example code, tools, and other stuff. When Evernote uses the word “SDK” they mean “library.”] So… why? Why do it if everyone has to do more work?

Seeking the least compromise to data-transfer performance, Evernote needed a solution that could shuffle large quantities of data with minimal overhead. Despite its superior efficiency over XML, REST still wasn’t good enough.

Whoa. REST has “superior efficiency over XML”? That’s just nonsense. REST is not a data format. REST is an architectural approach. REST does not mean “not XML”. If you want to transfer XML data using the REST approach, go ahead. That’s why Roy Fielding, Tim Berners-Lee, and Henrik F. Nielsen invented the Content-Type header. That’s what MIME types are for. You can transfer XML, or binary, or any sort of data with REST.

The implicit and incorrect assumption is that REST implies JSON, or that REST implies not binary. That’s false. There is no need to avoid REST in order to attain reasonable data transfer performance.

According to the article, that faulty reasoning is why Evernote selected Apache Thrift. Furthermore, as a benefit!! Thrift has tools to generate libraries for many platforms:

Thrift’s code-generating ability to write-once-and-deploy-to-many is also the reason Evernote is able to offer SDKs for so many platforms.

Yippee! But guess what! If you used REST, you wouldn’t need to generate all those libraries. And you’d have even broader platform support.

Just for fun, let’s have a look at the API that is being generated via Thrift. The Evernote API Reference looks like this:

OMG, the horror. Look at all that stuff. The reason people like REST is that they can figure out the data model just by perusing the URLs. It’s obviously not possible to do so in this case.

Evernote’s is not a modern API. It is a mass of complexity.

Not impressed.

Not Impressed

Must Love Clouds

Apigee is looking for a few good SEs.

An SE is a Sales Engineer. At Apigee, this is a member of the sales team who’s very technically adept, loves technology, and loves talking to other technologists about how to apply cool technology to solve hard problems.

This person needs to have a good developer background in server-side apps programming, with good C++, Java or C# skills, and probably one or more of Python, Ruby, PHP and NodeJS. Of course should know APIs, REST, SOAP very well, and should be handy with JSON, XML, and the various tools around those. Ought to know who Roy Fielding is, why jQuery is named jQuery, should know what RFC 2616 is, must love clouds, big data. Experience with AWS, Azure, RackSpace, VMware or IBM Smartcloud is a big plus. Beyond all that, the person’s got to love dealing with smart people with different perspectives.

If you know someone who fits this bill, lives near one of the NFL cities in the USA (you know, New York, Boston, Atlanta, Dallas, Houston, Chicago, Denver, San Fran, Los Angeles, etc), and wants to work for an ambitious late stage startup, send em my way: @dpchiesa on twitter.

The way Azure should have done it – A better Synonyms Service

This is a followup from my previous post, in which I critiqued the simple Synonyms Service available on the Azure Datamarket.

To repeat, the existing URI structure for the service is like this:

GET https://api.datamarket.azure.com/Bing/Synonyms/GetSynonyms?Query=%27idiotic%27

How would I do things differently?

The hostname is just fine – there’s nothing wrong with that. So let’s focus on the URI path and the other parts.

GET /Bing/Synonyms/GetSynonyms?Query=%27idiotic%27

Here’s what I would do differently.

  1. Simplify. The URI structure should be simpler. Eliminate Bing and GetSynonyms from the URI path, as they are completely extraneous. Simplify the query parameter. Eliminate the url-encoded quotes when they are not necessary. Result: GET /Synonyms?w=improved
  2. Add some allowance for versioning. GET /v1/Synonyms?w=positive
  3. Allow the caller to specify the API Key in the URI. (Eliminate the distorted use of HTTP Basic Auth to pass this information). GET /v1/Synonyms?w=easy&key=0011EEBB4477

What this gets you, as an API provider:

  1. This approach allows users to try the API from a browser or console without registering. The service could allow 3 requests per minute, or up to 30 requests per day, for keyless access. Allowing low-cost or no-cost exploration is critical for adoption.
  2. The query is as simple as necessary and no simpler. There is no extraneous Bing or GetSynonyms or anything else. It’s very clear from the URI structure what is being requested. It’s “real” REST.

What about multi-word queries? Easy: just URL-encode the space.
GET /v1/Synonyms?w=Jennifer%20Lopez&key=0011EEBB4477

There’s no need to add in url-encoded quotes for every query, in order to satisfy the 20% case where the query involves more than one word. In fact I don’t think multi-word would even be 20%. Maybe more like 5%.

For extra credit, do a basic content negotiation that looks at the incoming Accepts header and modifies the format of the result based on that header. As an alternative, you could include a suffix in the URI path, to indicate the desired output data format, as Twitter and the other big guys do it:

GET /v1/Synonyms.xml?w=adaptive&key=0011EEBB4477

GET /v1/Synonyms.json?w=adaptive&key=0011EEBB4477

As an API provider, conforming to a “pragmatic REST” approach means you will deliver an API that is immediately familiar to developers regardless of the platform they use to submit requests. That means you have a better chance to establish a relationship with those developers, and a better chance to deepen that relationship.

That’s why it’s so important to get the basic things right.

Azure Synonyms Service – How NOT to do REST.

Recently, I looked on the Azure data market place (or whatever it’s called) to see what sort of data services are available there. I didn’t find anything super compelling. There were a few premium, for-fee services that sounded potentially interesting but nothing that I felt like spending money on before I could try things out.

As I was perusing, I found a synonyms service. Nice, but this is nothing earth-shaking. There are already a number of viable, programmable synonyms services out there. Surely Thesaurus.com has one. I think Wolfram Alpha has one. Wordnik has one. BigHugeLabs has one that I integrated with emacs. But let’s look a little closer.

Let me show you the URL structure for the “Synonyms” service available (as “Community Technical Preview”!) on Azure.


https://api.datamarket.azure.com/Bing/Synonyms/GetSynonyms?Query=%27idiotic%27

Oh, Azure Synonyms API, how do I NOT love thee? Let me count the ways…

  1. There’s no version number. What if the API gets revised? Rookie mistake.
  2. GetSynonyms? Why put a verb in the URI path, when the HTTP verb “GET” is already implied by the request? Useless redundancy. If I call GET on a URI path with the word “Synonyms” in it, then surely I am trying to get synonyms, no?
  3. Why is the word Bing in there at all?
  4. Notice that the word to get synonyms of, must be passed with the query param named “Query”. Why use Query? Why not “word” or “term” or something that vaguely corresponds to the actual thing we’re trying to do here? Why pass it as a query param at all? Why not simply as part of the URL path?
  5. Also notice that the word must be enclosed in quotes, which themselves must be URL-encoded. That seems like an awkward design.
  6. What you cannot see in that URL is the authentication required. Azure says the authentication is “HTTP Basic Auth” which means you pass a username and password pair, joined by a colon then base64 encoded, as an HTTP Header. But… there is no username and password. Bing/Azure/Microsoft gives you an API Key, not a user name. And there’s no password. So you need to double the API key then base64 encode *that*, and pretend that it’s HTTP Basic Auth.

If readers aren’t persuaded that the above are evidence of poor API design, then you might consider heading over to the API Craft discussion group on Google Groups to talk it over.

Alternatively, or in addition, spend some time reading “the REST Manifesto,” Roy Fielding’s thesis paper, specifically chapter 5 in that document. It’s about 18 printed pages, so not too big a commitment.

The problem with releasing a poorly-designed API, is that it can do long-term damage.
As soon as a real developer takes a look at your service, he will not walk, he’ll RUN away to an alternative service. If your API is a pain to use, or is poorly designed, you are guaranteed to drive developers somewhere else. And they won’t come back! They might come just to poke around, but if they see a bad service, like this Synonyms service, they will flee, never to return. They will quickly conclude that you just don’t get it, and who could blame them?

So learn from Azure’s mistakes, and learn from the success of others. Take the time to get it right.

And now a word from my sponsor: Apigee offers a Rapid API Workshop service where we can send in experts to collaborate with your team on API design principles and practice. Contact us at sales@Apigee.com for more information.

APIs within the Enterprise – a Webinar

Recently I did a web chat with colleague Greg Brail discussing the use of APIs in the Enterprise.

Quick summary: SOA has been used with success within enterprises to interconnect systems. APIs address a different set of problems, and there is real value to be gained by using APIs to interconnect systems within the enterprise, as well as to provide external or partner access into enterprise systems.

Azure gets a well-deserved REST

In case you had any doubts of Programmable web’s data showing REST dominating other web API protocols, Microsoft, one of the original authors of SOAP, is fully embracing REST as the strategic web protocol for administering and managing Windows Azure services.

From Gigaom:

The new REST API that controls the entire system is completely rewritten, sources said.  ”Prior to this release, the Azure APIs were inconsistent. There was no standard way for developers to integrate their stuff in. That all changes now,” said one source who has been working with the API for some time and is impressed.

If you had 2 hours to spend learning stuff about web API protocols, spend 3 minutes understanding SOAP, and the balance on REST.

 

API Growth, SOAP v REST, etc

From HighScalability.com, John Musser’s GlueCon slides.  Interesting data pulled from ProgrammableWeb.com .   As I understand it, ProgrammableWeb is mostly a repository of APIs. It’s free to register and I don’t believe there is any sort of authentication – in other words anyone can post any web API.  Each listing includes an API endpoint, a service.

The main point Musser makes is that APIs are continuing to grow exponentially.  He didn’t go so far as to coin a new law (“Musser’s Law”) for the growth rate, but the trend is pretty certain.

Some other interesting takeaways from the slides:

  • REST v SOAP is one of the themes.  PW’s data shows REST waaay outpacing SOAP and other alternatives.


(For the REST purists who say REST is an architecture and SOAP is a set of envelope standards, my response is, yeah, but… we all know that’s not really true. ) 

  • Musser shows how gosh-darned simple it is to express “Get the current IBM share price” in REST, vs how complex it is in SOAP.  This is true, but misleading if you are not careful.  Lots of API calls require a small set of input parameters. Some do not.  For a counter example look at the complexity of posting a Twitter messaging using OAuth, which I wrote about previously.
  • There are companies doing more than a billion API transactions per day.  Twitter, Google, Facebook – you figured those. Also AccuWeather, Sabre, Klout, NetFlix.  Those are just the ones we know about.

I remember registering for Programmable Web about 10 years ago.  There were 4 or 5 online book-sellers in those days, each had different prices. I put together a book price quote service, which would accept an ISBN, then screen-scrape HTML from each of them, and respond with three or four prices – not all vendors listed all books.  It was just a novel example.  I used it in demonstrations to show what an API could do.  It was never used for real commerce.

I published it.  The screen scraping was a brittle joke.  Every time Amazon updated their HTML, the service would break, and people would email me complaining about it.  Though that service died a long while ago, it remains listed on ProgrammableWeb today. Because of that I think it’s important to consider the real implications of the PW numbers – not all the listed services are alive today, and many of them are not “real” even if they are alive.

Even with all that, the steepening upward trend does show the rapid ramp up of API development and usage.  Even if those APIs are not real, there are real developers building and testing them.  And real companies making a business of it (Apigee, to name one).  Also,  it’s pretty clear that JSON is winning.

One can gain only so much insight from viewing slides. It would have been nice to hear Mr Musser’s  commentary as well.

Side note – The term “API” used to denote a programmer’s interface exposed in a library.  Developers would use an API by linking with a library.  The term has silently expanded its meaning to include application message protocols – in other words, how to format and send messages to a service across a network.  These things are very different in my mind, but yet the same term, API, is used to describe them both, these days.

 

Apigee’s Best Practices for REST API design

I just read Apigee’s paper on pragmatic RESTful API design.

Very sensible, practical guidance. Good stuff for organizations confronting the REST phenomenon.  There are obviously many REST-based interfaces out there. Facebook, Google, Digg, Reddit, LinkedIn are just a few of the more visible services, coincidentally all social networks, that support REST.  But of course there is real value for enterprises in exposing resources in the same way. Wouldn’t it be nice if public records would be exposed by your municipal government via REST?  How many times have you wanted the data from a hosted app – what we used to call “application service provider” – in a machine-comprehensible format, instead of in an HTML page?

It’s worth examining the results the pioneers have achieved, to benefit from their experience.

As pioneers rushing to market, the designers of these early social network APIs may have sacrificed some quality in design, for speed of delivery.  Understandable. Apigee’s paper critiques some of those designs, and describes some of the rough edges. It’s like sitting in on a design review – and it’s an excellent way to learn.

Once you “get” REST, it all makes sense. It falls into place and the design principles and guidance offered by Apigee will seem like second nature. But for those grappling with a novel problem, it’s good to have a firm foundation from which to start.