hostgator coupon July 2014 hostgator coupon 2014 June Hostgator coupon June 2014 Dreamhost promo code 2014 our business news > first class business
Joseph Smarr » Implementing OAuth is still too hard… but it doesn’t have to be

Main menu:

Add to Google

Subscribe via e-mail:

insurance types cosmetics today fitness animals automobile reviews business money buy jewellery finance loans home helps> insurance companies in market finance news medical product money us technology in finance time roof repairing places where cook recipes business ideas business pay buy insurance car price in compamy marketing company guide cosmetics product diet info healthy live samples of business plan home decoration tech news auto insurance home improvement online business tips personal loans product reviews security types top business list top company list home tricks weight loss help what kind of business should i start business letter format business case template general business business type it security what is a good business to start health loss business me magazine news

business ideas

Site search


February 2009
« Dec   Mar »


Implementing OAuth is still too hard… but it doesn’t have to be

I recently helped Dave Winer debug his OAuth Consumer code, and the process was more painful than it should have been. (He was trying to write a Twitter app using their beta OAuth support, and since he has his own scripting environment for his OPML editor, there wasn’t an existing library he could just drop in.) Now I’m a big fan of OAuth–it’s a key piece of the Open Stack, and it really does work well once you get it working. I’ve written both OAuth Provider and Consumer code in multiple languages and integrated with OAuth-protected APIs on over half a dozen sites. I’ve also helped a lot of developers and companies debug their OAuth implementations and libraries. And the plain truth is this: it’s empirically way too painful still for first-time OAuth developers to get their code working, and despite the fact that OAuth is a standard, the empirical “it-just-works-rate” is way too low.

We in the Open Web community should all be concerned about this, since OAuth is the “gateway” to most of the open APIs we’re building, and quite often this first hurdle is the hardest one in the entire process. That’s not the “smooth on-ramp” we should be striving for here. We can and should do better, and I have a number of suggestions for how to do just that.

To some extent, OAuth will always be “hard” in the sense that it’s crypto–if you get one little bit wrong, the whole thing doesn’t work. The theory is, “yeah it might be hard the first time, but at least you only have to suffer that pain once, and then you can use it everywhere”. But even that promise falls short because most OAuth libraries and most OAuth providers have (or have had) bugs in them, and there aren’t good enough debugging, validating, and interop tools available to raise the quality bar without a lot of trial-and-error testing and back-and-forth debugging. I’m fortunate in that a) I’ve written and debugged OAuth code so many times that I”m really good at it now, and b) I personally know developers at most of the companies shipping OAuth APIs, but clearly most developers don’t have those luxuries, nor should they have to.

After I helped Dave get his code working, he said “you know, what you manually did for me was perfect. But there should have been a software tool to do that for me automatically”. He’s totally right, and I think with a little focused effort, the experience of implementing and debugging OAuth could be a ton better. So here are my suggestions for how to help make implementing OAuth easier. I hope to work on some or all of these in my copious spare time, and I encourage everyone that cares about OAuth and the Open Stack to pitch in if you can!

  • Write more recipe-style tutorials that take developers step-by-step through the process of building a simple OAuth Consumer that works with a known API in a bare-bones, no-fluff fashion. There are some good tutorials out there, but they tend to be longer on theory and shorter on “do this, now do this, now you should see that, …”, which is what developers need most to get up and running fast. I’ve written a couple such recipes so far–one for becoming an OpenID relying party, and one for using Netflix’s API–and I’ve gotten tremendous positive feedback on both, so I think we just need more like that.
  • Build a “transparent OAuth Provider” that shows the consumer exactly what signature base string, signature key, and signature it was expecting for each request. One of the most vexing aspects of OAuth is that if you make a mistake, you just get a 401 Unauthorized response with little or no debugging help. Clearly in a production system, you can’t expect the provider to dump out all the secrets they were expecting, but there should be a neutral dummy-API/server where you can test and debug your basic OAuth library or code with full transparency on both sides. In addition, if you’re accessing your own user-data on a provider’s site via OAuth, and you’ve logged in via username and password, there should be a special mode where you can see all the secrets, base strings, etc. that they’re expecting when you make an OAuth-signed request. (I plan to add this to Plaxo, since right now I achieve it by grepping through our logs and them IMing with the developers who are having problems, and this is, uhh, not scalable.)
  • Build an OAuth validator for providers and consumers that simulates the “other half” of the library (i.e. a provider to test consumer-code and a consumer to test provider-code) and that takes the code through a bunch of different scenarios, with detailed feedback at each step. For instance, does the API support GET? POST? Authorization headers? Does it handle empty secrets properly? Does it properly encode special characters? Does it compute the signature properly for known values? Does it properly append parameters to the oauth_callback URL when redirecting? And so on. I think the main reason that libraries and providers so far have all had bugs is that they really didn’t have a good way to thoroughly test their code. As a rule, if you can’t test your code, it will have bugs. So if we just encoded the common mistakes we’ve all seen in the field so far and put those in a validator, future implementations could be confident that they’ve nailed all the basics before being released in the wild. (And I’m sure we’d uncover more bugs in our existing libraries and providers in the process!)
  • Standardize the terms we use both in our tutorials and libraries. The spec itself is pretty consistent, but it’s already confusing enough to have a “Consumer Token”, “Request Token”, and “Access Token”, each of which consist of a “Token Key” and “Token Secret”, and it’s even more confusing when these terms aren’t used with exact specificity. It’s too easy to just say “token” to mean “request token” or “token” to mean “token key”–I do it all the time myself, but we really need to keep ourselves sharp when trying to get developers to do the right thing. Worse still, all the existing libraries use different naming conventions for the functions and steps involved, so it’s hard to write tutorials that work with multiple libraries. We should do a better job of using specific and standard terms in our tutorials and code, and clean up the stuff that’s already out there.
  • Consolidate the best libraries and other resources so developers have an easier time finding out what the current state-of-the-art is. Questions that should have obvious and easily findable answers include: is there an OAuth library in my language? If so, what’s the best one to use? How much has it been tested? Are their known bugs? Who should I contact if I run into problems using it? What are the current best tutorials, validators, etc. for me to use? Which companies have OAuth APIs currently? What known issues exist for each of those providers? Where is the forum/mailing-list/etc for each of those APIs? Which e-mail list(s) should I send general OAuth questions to? Should I feel confident that emails sent to those lists will receive prompt replies? Should I expect that bug reports or patches I submit there will quickly find their way to the right place? And so on.
  • Share more war stories of what we’ve tried, what hasn’t worked, and what we had to do to make it work. I applauded Dave for suffering his developer pain in public via his blog, and I did the same when working with Netflix’s API, but if we all did more of that, our collective knowledge of bugs, patterns, tricks, and solutions would be out there for others to find and benefit form. I should do more of that myself, and if you’ve ever tried to use OAuth, write an OAuth library, or build your own provider, you should too! So to get things started: In Dave’s case, the ultimate problem turned out to be that he was using his Request Secret instead of his Access Secret when signing API requests. Of course this worked when hitting the OAuth endpoint to get his Access token in the first place, and it’s a subtle difference (esp. if you don’t fully grok what all these different tokens are for, which most people don’t), but it didn’t work when hitting a protected API, and there’s no error message on any provider that says “you used the wrong secret when signing your request” since the secrets are never transmitted directly. The way I helped him debug it was to literally watch our debugging logs (which spit out all the guts of the OAuth signing process, including Base String, Signature Key, and final Signature), and then I sent him all that info and asked him to print out the same info on his end and compare the two. Once he did that, it was easy to spot and fix the mistake. But I hope you can see how all of the suggestions above would have helped make this process a lot quicker and less painful.

What else can we as a community do to make OAuth easier for developers? Add your thoughts here or in your own blog post. As Dave remarked to me, “the number of OAuth developers out there is about to skyrocket” now that Google, Yahoo, MySpace, Twitter, Netflix, TripIt, and more are providing OAuth-protected APIs. So this is definitely the time to put in some extra effort to make sure OAuth can really achieve its full potential!

  • David Recordon

    Makes sense and I think that these lessons and ideas are true for a lot of technologies. If the OAuth community were to attack this list, what order do you think makes sense? Is a validator the most important thing to build first?

  • Joseph Smarr

    Actually I think the first step should be a good recipe guide and a companion “transparent provider” that the recipe uses. It would take the user through getting/writing a library, hitting a basic “hello world” up to a more complex API call, and it would provide lots of detail and feedback at each step. The idea is, we could tell developers to “first get your code working against this extra-friendly OAuth API, and then turn it to twitter or whoever else you want o hit, since then you'll be confident that it at least works in theory already.” And yes, I agree that these types of developer aids are necessary for all the Open Stack building blocks!

  • David Recordon

    A bit like your OpenID recipe that you wrote a few years ago. We're planning to write this sort of guide for our book, but haven't started on the OAuth content yet.

  • Ryan Kennedy

    Joseph, I totally agree. As someone who works on OAuth code every day (mostly working on SDKs to access Yahoo!'s OAuth protected social resources), OAuth seems pretty simple to me. However, as we've all seen, users don't have the same experience. Consumers are buggy, providers are buggy and OAuth itself is just plain complicated to understand for someone who just wants to execute a bit of YQL that performs an image search.

    Feel free to get in touch with me, I'd love to help in any way I can. I'm particularly interested in the transparent OAuth provider and OAuth validator. Given my role at Yahoo!, I might even be able to convince the powers that be that it's a good idea for me to spend some of my work time on such an effort.

  • Joseph Smarr

    Ryan-sounds great, and I'll definitely take you up on that offer! ;) It's clear to me that a little extra time and effort by people like you and me whose jobs and companies depend on OAuth being easy to make it so will be a wise investment indeed.

  • thekarladam

    I don't know, personally I never found OAuth hard so much as just new. The problems that people are facing aren't so much with the hows, as Winer had the hashing and such correct, but didn't understand that concepts as to why certain things were being done. I grokked OAuth after looking at the Yahoo! OAuth guide which displayed a graphic of the handshake process and was clear to me how to move from unauthorized to authorized and authenticated.

  • dave

    Why don't we have a dinner in March to discuss?

  • lmorchard

    A “transparent OAuth Provider” is exactly what I was looking for when I started trying to help Dave, too. I've not yet had the chance until now to dive very deep into OAuth myself, and it's very easy to get one thing inexplicably wrong (eg. not url-encoding all the right parts) and have no clue what you got wrong because the signatures are necessarily so opaque.

    Would be nice if the “training wheels” of a transparent provider could take all the client-side key/secret and duplicate the expected work for the client side as a cross-check.

  • Luca

    I think that all these ideas make perfectly sense, it would be ideal if recipes were collected for all of the OAuth libraries (i mean: having a version of the recipe for each library out there…).

    In the meantime, today I have started to work on a prototype “transparent oauth provider”, I should have something usable by tomorrow evening (day-job permitting :-) )

  • Daniel Rubio

    I followed your comment on Dave Winer post.
    Just thought I would mention it. The Google OAuth playground is the one thing that finally got me to understand the process, since its interactive showing the requests, headers, tokens and responses.
    Its located here :

  • dready

    Thanks for presenting this action list. I agree that these will tremendously improve the developer accessibility for OAuth.

    Regarding terminology confusion, that's very true with the sheer number of tokens coupled with possibly name collision with app-specific parameters (which could be called “api_key”, or “secret”). It makes it so easy to get a typo or function parameters swapped. On FireEagle, two different access tokens (user-specific and general access) are used, which may make things worse.

    I haven't dabbled in OAuth much, but my first experience was implementing the Fire Eagle widget on MojiPage. We use the Python OAuth library, and provide useful abstraction for the widget (also written in Python) so that it needs only call a few functions. Debugging was a tad harder than the usual web app, but it was a smooth ride on the whole.

  • Anonymous

    The explanation as to where to include variables, whether it be in the header, or whether it be in the post, or what-have you, its very cryptic and frustrating as everyone seems to think the concept of hashing and passing back keys is the difficult concept. It’s not, formatting the requests properly is and so far I have not come across a decent explanation or even a fully reasonable testing ground with one exception based on a single twitter implementation.

    Hashing is handled by the OS, there is nothing difficult about it.

    I currently see it as a general failure on the part of the people doing the dumbing down in their explanations while failing to provide the specific details on their formatting for specific functions.

    I am sure my opinion will change when I am done sorting through pages of code to figure out what it was the guy writing the service instructions actually meant to say.

    Meanwhile, facebook connect explodes, and everybody who thought an overly complex protocol versus a straightforward secure process was the right path to choose can milk another fat paycheck.

  • Part time jobs

    Very nice article about implementing oath

  • Russs

    It’s now June of 2011 and I’m finding it very difficult to get an api interface figured out!

  • chengwenxia

    Chanel Factory
    Chanel Outlet
    Chanel Factory Outlet
    Chanel Factory Store
    chanel handbags outlet
    chanel handbags outlet online
    chanel factory outlet store
    replica chanel watches
    replica chanel sunglasses
    cheap chanel bags
    chanel handbags low price
    Chanel 2.55 Classic Handbags
    Replica Chanel 2.55 Classic Handbags
    Chanel Factory Outlet
    replica Chanel 2.55 bags
    fake Chanel 2.55 Classic handbags
    Classic Chanel 2.55 handbags knockoff
    Chanel 2.55 Reissue Handbags
    Chanel Factory Outlet
    Replica Chanel 2.55 Reissue Handbags
    fake Chanel 2.55 Reissue handbags
    hanel 2.55 handbags knockoff
    replica Chanel 2.55 Reissue bags
    Replica Chanel Belts
    Replica Chanel Leather Belts
    fake chanel Belts
    chanel Belts knockoff
    chanel Belts knockoff from china
    Replica Chanel Leather Belts Wholesale
    Replica Chanel Cambon Handbags
    Chanel Factory Outlet
    replica Chanel Cambon bags
    fake Chanel Cambon handbags
    Chanel Cambon handbags knockoff
    Replica Chanel Clutch Handbags
    Chanel Factory Outlet
    replica Chanel Clutch bags
    fake Chanel Clutch handbags
    Chanel Clutch handbags knockoff
    Chanel Clutch handbags knockoff from china
    Replica Chanel Coco Handbags
    Chanel Factory Outlet
    replica Chanel Coco bags
    fake Chanel Coco handbags
    Chanel Coco handbags knockoff
    Chanel Coco handbags knockoff from china
    Chanel Handbags 2010 Replica
    Cheap Chanel Handbags 2010
    replica Chanel Handbags 2010
    fake Chanel Handbags 2010
    Chanel bags 2010 knockoff
    Chanel bags 2010 knockoff from china
    Chanel Handbags 2011 Replica
    Cheap Chanel Handbags 2011
    replica Chanel Handbags 2011
    fake Chanel Handbags 2011
    Chanel bags 2011 knockoff
    Chanel bags 2011 knockoff from china
    Chanel Handbags 2012 Replica
    Cheap Chanel Handbags 2012
    replica Chanel Handbags 2012
    fake Chanel Handbags 2012
    Chanel bags 2012 knockoff
    Chanel bags 2012 knockoff from china
    Cheapest Chanel Ipad 2 Cases
    Discount Chanel Ipad 2 Case
    replica Chanel Ipad 2 Case
    fake Chanel Ipad 2 Covers
    Chanel Ipad 2 Case knockoff
    Chanel Ipad 2 Case knockoff from china
    Cheapest Chanel Iphone 4 Cases
    Discount Chanel Iphone 4 Case
    replica Chanel Iphone 4 Covers
    fake Chanel Iphone 4 Case
    Chanel Iphone 4 Case knockoff
    Chanel Iphone 4 Case knockoff from china
    Cheap Chanel Ladies Watches
    Replica Chanel Ladies Timepieces
    Chanel Ladies Watches replica
    fake Chanel Ladies Watches
    Chanel Ladies Watches knockoff
    Chanel Ladies Watches knockoff from china
    Cheap Chanel Mens Watches
    Replica Chanel Mens Timepieces
    Chanel Mens Watches replica
    fake Chanel Mens Watches
    Chanel Mens Watches knockoff
    Chanel Mens Watches knockoff from china
    Replica Chanel Mini Handbags
    Chanel Factory Outlet
    replica Chanel Mini bags
    fake Chanel Mini handbags
    Chanel Mini handbags knockoff
    Chanel Mini handbags knockoff from china
    Chanel Sunglasses 2012
    Chanel Sunglasses 2012 Sale
    Cheap Chanel Sunglasses
    Cheap Chanel Sunglasses Online
    replica Chanel Sunglasses 2012
    fake 2012 Chanel Sunglasses
    Chanel Sunglasses knockoff from china
    Replica Chanel Tote Handbags
    Chanel Factory Outlet
    replica Chanel Tote bags
    fake Chanel Tote handbags
    Chanel Tote handbags knockoff
    Chanel Tote handbags knockoff from china
    Cheap Replica Chanel Wallets
    Chanel Wallets Outlet
    Chanel Wallets Outlet Store
    replica Chanel Purses
    fake Chanel Wallets
    Chanel Wallets knockoff
    Chanel Wallets knockoff from china
    Cheap Chanel Bags 2012
    Cheap Chanel Bags Replica
    replica Cheap Chanel Bags
    fake Cheap Chanel Bags
    Cheap Chanel Bags knockoff
    Cheap Chanel Bags knockoff from china
    chanel bags
    chanel bags replica
    chanel handbags
    Replica Chanel Bags
    Cheap Chanel Handbags
    Chanel Bags Sale
    Chanel Handbags Store
    discount chanel bags
    chanel bags outlet
    cheap chanel bag
    fake handbags chanel
    chanel bags online
    chanel bags free shipping
    chanel handbags free shipping
    Replica Chanel Bags Online