Joseph Smarr

Thoughts on web development, tech, and life.

Category: Personal (page 1 of 3)

Dreaming big


I’ve recently been working on a side project with my good friend Chris Lamb to scale up Google’s Deep Dream neural net visualization code to operate on giant (multi-hundred megapixel) images without crashing or taking an eternity. We recently got it working, and our digital artist friend (and fellow Plaxo alum!) Dan Ambrosi has since created some stunning work that’s honestly exceeded all of our expectations going in. I thought it would be useful to summarize why we did this and how we managed to get it working.

Even if you don’t care about the technical bits, I hope you’ll enjoy the fruits of our labor. :)

The ‘danorama’ back story

gridDan’s been experimenting for the past several years with computational techniques to create giant 2D-stitched HDR panoramas that, in his words, “better convey the feeling of a place and the way we really see it.” He collects a cubic array of high-resolution photos (multiple views wide by multiple views high by several exposures deep). He then uses three different software packages to produce a single seamless monster image (typically 15-25k pixels wide): Photomatix to blend the multiple exposures, PTGui to stitch together the individual views, and Photoshop to crop and sweeten the final image. The results are (IMO) quite compelling, especially when viewed backlit and “life size” at scales of 8’ wide and beyond (as you can do e.g. in the lobby of the Coastside Adult Day Health Center in Half Moon Bay, CA).

“I’d like to pick your brain about a little something…”

Dan AmbrosiWhen Google first released its deep dream software and corresponding sample images, everyone went crazy. Mostly, the articles focused on how trippy (and often disturbing) the images it produced were, but Dan saw an opportunity to use it as a fourth tool in his existing computational pipeline–one that could potentially create captivating impressionistic details when viewed up close without distorting the overall gestalt of the landscape when viewed at a distance. After trying and failing to use the code (or any of the DIY sites set up to run the code on uploaded images) on his giant panoramas (each image usually around 250MB), he pinged me to ask if I might be able to get it working.

I had no particular familiarity with this code or scaling up graphics code in general, but it sounded like an interesting challenge, and when I asked around inside Google, people on the brain team suggested that, in theory, it should be possible. I asked Chris if he was interested in tackling this challenge with me (both because we’d been looking for a side project to hack on together and because of his expertise in CUDA, which the open source code could take advantage of to run the neural nets on NVIDIA GPUs), and we decided to give it a shot. We picked AWS EC2 as the target platform since it was an easy and inexpensive way to get a linux box with GPUs (sadly, no such instance types are currently offered by Google Compute Engine) that we could hand off to Dan if/when we got it working. Dan provided us with a sample giant panorama image, and off we set.

“We’re gonna need a bigger boat…”

18-KolobCanyon_megoHDR60-1600Sure enough, while we could successfully dream on small images, as soon as we tried anything big, lots of bad things started happening. First, the image was too large to fit in the GPU’s directly attached memory, so it crashed. The neural nets are also trained to work on fixed-size 224×224 pixel images, so they had to downscale the images to fit, resulting in loss of detail. The solution to both problems (as suggested to me by the deep dream authors) was to iteratively select small sub-tiles of the image and dream on them separately before merging them back into the final image. By randomly picking the tile offsets each time and iterating for long enough, the whole image gets affected without obvious seams, yet each individual dreaming run is manageable.

Once we got that working, we thought we were home free, but we still couldn’t use the full size panoramas. The GPUs were fine now, but the computer itself would run out of RAM and crash. We thought this was odd since, as mentioned above, even the largest images were only around 250MB. But of course that’s compressed JPEG, and the standard Python Imaging Library (PIL) that’s used in this code first inflates the image into an uncompressed 2D array where each pixel is represented by 3×32 bits (one per color channel), so that the same image ends up needing 3.5GB (!) of RAM to represent. And then that giant image is copied several more times by the internal code, meaning even our beefiest instances were getting exhausted.

So we set about carefully profiling the memory usage of the code (and the libraries it uses like NumPy) and looking for opportunities to avoid any copying. We found the memory_profiler module especially helpful, as you can annotate any suspicious methods with @profile and then run python -m memory_profiler to get a line-by-line dump of incremental memory allocation. We found lots of places where a bit of rejiggering could save a copy here or there, and eventually got it manageable enough to run reliably on EC2’s g2.8xlarge instances. There’s still more work we could do here (e.g. rewriting numpy.roll to operate in-place instead of copying), but we were satisfied that we could now get the large images to finish dreaming without crashing.

BTW, in case you had any doubts, running this code on NVIDIA GPUs is literally about 10x faster than CPU-only. You have to make sure caffe is compiled to take advantage of GPUs and tell it explicitly to use one during execution, but trust me, it’s well worth it.

Solving the “last mile” problem

new-dreamWith our proof-of-concept in hand, our final task was to package up this code in such a way that Dan could use it on his own. There are lots of tweakable parameters in the deep dream code (including which layer of the deep neural net you use to dream, how many iterations you run, how much you scale the image up and down in the process, and so on), and we knew Dan would have to experiment for a while to figure out what achieved the desired artistic effect. We started by building a simple django web UI to upload images, select one for dreaming, set the parameters, and download the result. The Material Design Lite library made it easy to produce reasonably polished looking UI without spending much time on it. But given how long the full images took to produce (often 8-10 hours, executing a total of 70-90 quadrillion (!) floating point operations in the process), we knew we’d like to include a progress bar and enable Dan to kick off multiple jobs in parallel.

Chris took the lead here and set up celery to queue up dispatching and controlling asynchronous dreaming jobs routed to different GPUs. He also figured out how to multiply together all the various sub-steps of the algorithm to give an overall percentage complete. Once we started up the instance and the various servers, Dan could control the entire process on his own. We weren’t sure how robust it would be, but we handed it off to him and hoped for the best.

“You guys can’t believe the progress i’m making”

dreamingOnce we handed off the running EC2 instance to Dan, we didn’t hear much for a while. But it turned out that was because he was literally spending all day and night playing with the tools and honing his process. He started on a Wednesday night, and by that Saturday night he messaged us to say, “You guys can’t believe the progress I’m making. I can hardly believe it myself. Everything is working beautifully. If things continue the way they are, by Monday morning I’m going to completely amaze you.” Given that we’d barely gotten the system working at all, and that we still really didn’t know whether it could produce truly interesting output or not, this was certainly a pleasant surprise. When we probed a bit further, we could feel how excited and energized he was (his exact words were, “I’m busting, Jerry, I’m busting!”). It was certainly gratifying given the many late nights we’d spent getting to this point. But we still didn’t really know what to expect.

The following Monday, Dan unveiled a brand new gallery featuring a baker’s dozen of his biggest panoramic landscapes redone with our tool using a full range of parameter settings varying from abstract/impressionistic to literal/animalistic. He fittingly titled the collection “Dreamscapes”. For each image, he shows a zoomed-out version of the whole landscape that, at first glance, appears totally normal (keep in mind the actual images are 10-15x larger in each dimension!). But then he shows a series of detail shots that look totally surreal. His idea is that these should be hung like giant paintings 8-16’ on a side. As you walk up to the image, you start noticing the surprising detail, much as you might examine the paint, brush strokes, and fine details on a giant painting. It’s still hard for me to believe that the details can be so wild and yet so invisible at even a modest distance. But as Dan says in his intro to the gallery, “we are all actively participating in a shared waking dream. Science shows us that our limited senses perceive a tiny fraction of the phenomena that comprise our world.” Indeed!

From dream to reality

galleryWhile Dan is still constantly experimenting and tweaking his approach, the next obvious step is to print several of these works at full size to experience their true scale and detail. Since posting his gallery, he’s received interest from companies, conferences, art galleries, and individuals, so I’m hopeful we’ll soon be able to see our work “unleashed” in the physical world. With all the current excitement and anxiety around AI and what it means for society, his work seems to be striking a chord.

Of course the confluence of art and science has always played an important role in helping us come to terms with the world and how we’re affecting it. When I started trying to hack on this project in my (copious!) spare time, I didn’t realize what lay ahead. But I find myself feeling an unusual sense of excitement and gratitude at having helped empower an artistic voice to be part of that conversation. So I guess my take-away here is to encourage you to (1) not be afraid to try and change or evolve open source software to meet a new need it wasn’t originally designed for, and (2) don’t underestimate the value in supporting creative as well as functional advances.

Dream on!

Has Tesla’s personalization crossed the line?

One of the cool things about owning a Tesla is that when you have to take it in for service, they often give you a loaner Tesla to take home. It feels like nothing’s changed–until you realize that all of your personalized settings have effectively been reset. In the past, this wasn’t a big deal, but I just took my car in for 36 hours and had an unexpectedly jarring experience. I thought I’d share it because it surprised and fascinated me how pervasive and important this personalization has become.

It may be tempting to dismiss this all as “first world problems”, and obviously the car still did go from A to B just fine, but I was struck by how much I’d unknowingly come to depend on many forms of personalization Tesla offers. None of the following examples are hypothetical or contrived–they each took me by surprise and made me go “hmm”:

  1. I couldn’t open my garage door when I got home (I don’t normally carry a house key since I come and go by car, but luckily the garage has a keypad backup!)
  2. My seat position, steering mode (stiffness), etc. were different enough to frustrate my muscle memory
  3. The loaner hadn’t enabled “walk away door locks”, so I inadvertently left the car unlocked for several hours
  4. I had to re-pair my phone via bluetooth before I could resume listening to my current audible audiobook (never as smooth an experience as it should be, esp. when adding a duplicate “Tesla Model S” endpoint!)
  5. My home/work/favorite map destinations were missing, meaning I had to re-find and enter my daughter’s preschool address (to get traffic-aware driving directions/time)
  6. I was surprised to find my windshield and windows fogged up in the morning, until I remembered that my car normally remembers when I tend to leave in the morning and automatically warms itself up beforehand (“smart preconditioning”)
  7. Similarly, my car normally notifies me proactively if there’s a faster route to home/work than usual (based on current traffic conditions), but not this time
  8. My phone buzzed from the Tesla app, but it was because they’d stopped charging my car at the service center, not the car I was currently charging at work
  9. I can normally find where I parked my car, see how much charge it has, and even unlock and start it using the Android app, but none of those things work with the loaner car
  10. My daughter loves listening to some of the kids music stations on Slacker radio (each car comes with a pre-registered account), but the loaner car didn’t have any account attached, meaning I couldn’t play her that station (and even if I could, it wouldn’t remember which songs we’d ranked up/down before)

By the time I swapped back, I’d manually corrected many of these differences, but of course that creates its own problem–this loaner car was now full of my personal data (home, work, daughter’s school, paired to my phone, etc.) with no obvious way of wiping the data (I presume they do something like a factory reset before handing out the car to the next customer, but it was never discussed with me).

Now, any loaner car situation would entail some of the problems above, but most cars aren’t as personalized to start with, and the fact that I was still driving “the same car” made it all the more surprising when the car didn’t behave like it used to. More to the point, Tesla’s persistent internet connection and over-the-air software updates put it in a unique position to actually solve many or all of the above problems. I normally think of cars as fairly fungible, but clearly these are not normal cars.

“You can’t go this far and then not go further!” –Homer Simpson

To me, Tesla’s personalization has clearly crossed the line from a minor convenience to an essential part of the ownership experience. The solution then, it seems to me, is to take Homer’s advice and sync all your personalized settings to the cloud, thus enabling you to “log in” to a car and transfer your settings before you start driving. Then you could also “log out” when you’re done and reset the car. Tesla already has all the necessary building blocks to build this–the car is persistently connected to the Internet, each driver has a “My Tesla” account that knows the VIN number(s) of their car(s), and the Tesla mobile app lets you log in with your My Tesla account and view/control the state of your car.

They even dipped their toe in the water with their recent calendar integration, which uses bluetooth and your My Tesla app to show your upcoming calendar appointments on the Tesla console (with one-tap driving directions to the destinations). It’s an effective but oddly orchestrated integration, suggesting that they were aware of the potential to “do it all in the cloud”, but chose to stop short (perhaps for a mix of technical and privacy reasons?)

Tesla also currently has support for multiple “driver profiles” per vehicle, which control not only the seat positions but also favorite radio/streaming stations, preferred steering mode, A/C and temperature, and more. It’s not much of a leap to suggest that you should be able to enter a new Tesla, pair your phone (perhaps with a quick tap on an NFC terminal built into the center console?) and choose to “download your driver profile”, bringing with it all of the personalized settings mentioned above. This would be useful not only for loaner cars but also when upgrading your own car or even borrowing a friend’s car. Since not all cars have identical hardware configurations, this “import” step would allow each car to flexibly pull down the applicable settings while ignoring the rest. Such a model would also be a lot simpler than attempting a full two-way sync.

As a bonus, this might solve a silly but common frustration / feature request I’ve heard from multiple Tesla owners: the ability to select your driver profile from the mobile app before entering the car. Anyone with a significantly shorter spouse often has to embark on a series of contortions to reach the center console when the seat was  left all the way forward. :) I presume this isn’t currently possible because the driver profiles are not synced to the cloud, but once they were…

To end on a personal note, one of the things I love most about using and working on cutting-edge technology is the feeling of how “unfinished” it is. Swimming in uncharted waters reminds me that the future has yet to be written, and that it’s up to all of us to figure out where to head next. I didn’t realize that my car had gotten sufficiently personalized to need a cloud-based auth system, but it now seems obvious in hindsight. What other needs and opportunities will we discover as the lines between digital technology and the physical world continue to blur? Only one way to find out…

Turns out we still need Plaxo (or something like it)

Despite having helped build a startup 12 (!) years ago focused on keeping your address book up to date, despite the massive intervening adoption of smart phones and social networks, and despite being connected to an unusually tech-savvy cohort of friends and relatives, my address book is still a disastrous mess. Sound familiar? This thought always lingered in my head, but it confronted me full-force this weekend while my wife and I struggled in vain to produce an updated mailing list to announce the birth of our new son. We’d done this for our last child (sadly using a shared Google spreadsheet in absence of any more purpose-built tool), so we were only 2-3 years out-of-date, yet an astonishing 45% of the addresses of our friends and family needed updating, and the task was not easily achieved.

This led me to ponder why this seemingly tractable problem remains so stubbornly unsolved, and what, if any, hope lies ahead.

The false saviors

It’s hard to remember that when we started Plaxo in 2002, there was no iPhone, no Facebook, not even Friendster or Flickr. Our biggest opportunity was writing a Microsoft Outlook plugin. Since then, many people have said that the problem of out-of-date contact info would soon be a relic of the past. Smartphones would keep us seamlessly connected. Social network profiles would always contain our latest info. Number portability, Gmail, and mainstream messaging services would create long-lasting identifiers. The death of snail mail would obviate the need for physical addresses. And so on. We often worried about these trends, and in some sense they have each helped us get more connected, yet they clearly haven’t solved the core problem. Why not? A few reasons…

Contact info still changes frequently (esp. in aggregate). As mentioned above, nearly half of the people we sent birth announcements to last time physically moved in the past 2-3 years. Now in our age range that’s probably more than usual (buying a first house, getting a new job, etc.), but it’s still staggeringly high. And it’s not just physical addresses. I’m constantly wondering “is this the right email address or cell phone number to use for so-and-so, or is it dead / rarely checked these days?” Ditto for “who do I know who still works at [insert company here]?” Even using circles in Google+, I’m often wondering “gee, is my Googlers circle still a safe place for Google-only discussions?” Even when the info hasn’t changed, I’m often unsure if it’s still current. And of course I keep meeting new people, many of whom I haven’t ever collected the relevant info for (even if they wouldn’t mind me having it).

Social networks suck at contact info. As surprising as the staleness of our address list was, I was even more surprised how few of my contacts had their latest address in Google+, Facebook, or elsewhere online. Many of these people used to work at Plaxo, and all are online using social networks daily. Yet the info is either missing, stale, or not shared with me. Why? In theory, social networks subsumed Plaxo-like functionality, but in practice there are deep reasons why they fall short.

  • Contact info is buried. I have to click to someone’s profile, click to their about/contact tab, scroll down, and then hope their info is shared with me. When’s the last time you viewed your own “about” tab (let alone someone else’s contact info section)? It’s out of sight, out of mind. In fact, even my own home address was out of date on Google+ and Facebook until long after I moved. It’s just not something you naturally think about while checking your news feed.
  • You don’t want everyone to have your info. Even though most social networks provide a way for you to share personal contact info, most users don’t want all of their “friends” to have all of their personal details. I recall hearing back in the day as Facebook grew that more people were actually deleting their contact info because they were making more and more loose-tie friends who they didn’t feel comfortable sharing that info with. On Google+ you can (of course) use circles to finely control who sees your home info, and Facebook has since followed suit, but as my wife put it when I asked her today, “eew, I don’t want to put my home address on Facebook”. You have to trust the site itself, your ability to navigate their privacy controls (and keep them up-to-date as your life changes), and the site’s ability to honor your choices before you’ll use social networks to share sensitive info. For most people, that bar has not yet been met.
  • Oh, and that whole “data portability” thing. When I want to see if I have someone’s latest home address, where should I look? My address book? Only if I can pull in the info shared with me from social networks. Not surprisingly, Google+ syncs to Google Contacts, but everything else is still a walled garden. You have to go knock on all the doors. For every person you care about. Every time. Even though they chose to share it with you. Sound broken? I agree. But I guess we weren’t loud enough.

Smartphones aren’t smart about contact info. Your smartphone address book does a good job of following you around from device to device and desktop to mobile. The only problem is that it is syncing garbage data. You’d think that using it to send and receive phone calls, text messages, emails, and looking up driving directions would make your address book fresher and more complete. But you’d be wrong. Chalk it up to app fragmentation and no one really trying hard to solve the problem in the first place, esp. at the OS level. Even at Google, the Gmail Contacts team is separate from the android “People app” team, and most OEMs bundle their own separate address book. Good luck.

No one’s helping you scrounge. Another fascinating if infuriating aspect of my recent Labor Day labor to update our mailing list was how often I could find the addresses by scrounging through my email archives, text messages, and manually entered address book info. I’d even been to many of the homes I lacked addresses for! In the absence of a good contact info syncing solution, most people still solve the problem “out of band” via existing communication channels (“leaving now, btw what’s your address in SF?” “it’s 123 Fake St., see you soon!”). Yet nothing is helping you extract, save, and aggregate that info for the next time you need it. It’s still a painful, manual process that you need to “get good at”. And this is for close ties–not random acquaintances–so it’s surely just an “entropy” problem, not a “stalking” problem.

The wisdom of crowds? Another trend I was sure would take off in the age of the cloud was some kind of solution for pooling contact info between families, friends, and organizations. At Plaxo we used to always have the problem, “Who’s got the cell phone for that new guy we just hired?” and the answer was always “someone in the room” (you just don’t know who in advance, and you have to ask them first). Many families still have a designated aunt who dutifully maintains the conglomerated birthday and holiday card list. And “hey Garret, remind me what Pete’s new address is?” still gets the job done in a pinch without offending. So why does each address book still start from scratch as if it were the only record of knowledge in the universe?

A new hope?

In the years since leaving Plaxo to help start Google+, I’ve talked to nearly every “address book 2.0” startup that’s reared its head. Apparently I’ve got a reputation in the Valley for not being “over” this contact info problem. Many have offered clever twists, but none have fundamentally addressed the challenges above. And perhaps unsurprisingly, as a result, none have saved the world from its frustrating fragmentation. So why am I still eternally optimistic? Because it’s a real, mainstream problem, there’s no good reason people want fragmentation to persist, and increasingly smartphones do participate in the events that collect or verify contact info. Plus there’s still the cloud opportunity for making each other smarter. So how might such a solution emerge?

When trying to solve any complex social problem, one good question to ask is, “What does the ideal solution state look like?” In the case of up-to-date contact info, I’d argue we still don’t know. You could say it’s everyone being on a single social network and using it perfectly, but is that ever going to be realistic? I’d say it’s more likely a mix of assisted sync and scrounge. In other words: help me collect whatever’s been shared with me via social networks or communication tools. And the place to do that is logically a smartphone (backed by a cloud-hosted account). Google or Apple are, in theory, in a great position to make progress on this, but I suspect it will be a startup that gets the job done, since it can be their sole focus and brand identity.

Such a startup would have to embrace the messy reality I’ve outlined above and turn it into a strength. Use all the available APIs and other tricks to help me find the contact info that has been shared with me. Keep track of when it was last updated (don’t make me guess). Parse through all my emails and texts for stuff that looks like contact info. Use my phone’s location history to ask me whose house I just visited when it doesn’t look like a business. Remind me what email or phone number each contact last used, and let me easily ping them back if I need some updated info. Help me build custom lists for things like holiday or birth announcements, and use that as an opportunity to ask for updated info. And partner with sites like TinyPrints not only to send those cards but also to send change-of-address cards when I myself move (something you should also be able to detect using my phone). Once you start gaining traction helping individuals keep their address books up-to-date, add a layer to pool it with family, friends, and colleagues in a privacy-preserving way (e.g. an easy way to see who knows someone’s phone number, but you still have to ask them to share it with you).

Is there enough here to build a successful business around? You be the judge. But is this still a real problem that real people still wish someone would solve? Abso-f*cking-lutely.

Personal Trainer: The Best Investment I’ve Made in Years

Growing up, I was pretty active physically. But after leaving home for college and pursuing a career in computers, my activity level continued to fall while my weight moved in the opposite direction. I tried numerous times to start working out regularly, but I would always fall off the wagon after a matter of weeks when life got extra busy. While I’d resisted the idea for years, last fall I finally decided to start working twice a week with a personal trainer, and the results have been fantastic–I now work out regularly and reliably (both with my trainer and on my own in between), I feel stronger, healthier, and happier, and my career is none the worse for spending the extra time I didn’t think I had.

I’ve told several colleagues this story (not surprisingly, many in Silicon Valley are in the same boat–working long hours and neglecting their health even though they’re aware that it’s a problem), and it’s inspired several of them to follow suit. And apparently I’m not alone. So I thought it would be worth recounting here both why this has worked so well and why it nevertheless took me far too long to convince myself to do it.

What’s so great about a personal trainer?

The biggest advantage by far of working with a personal trainer is that I actually go work out regularly now. That’s because my trainer puts our appointments on my calendar every week, waits for me to show up, and charges me money if I don’t. That may sound glib, but the core problem I’ve had for the past few years is that I couldn’t bring myself to work out regularly on my own. I’d get too busy, or there wouldn’t be a convenient time, or I’d be tired, or whatever, but the bottom line is it wouldn’t get done, and now it does. I like to think of myself as someone with a strong will, a good sense of discipline, and a rational, long-term outlook, but clearly I’m not the only one that finds it hard to “kick my own butt” into exercising regularly. Instead of worrying about exactly what exercises to do and how often, the “high-order bit” in fixing my situation was “just show up a few times a week and do anything“.

In retrospect, the solution is so obvious: success looks like you blocking off an hour on your calendar a few times a week to go to the gym, and doing that consistently for the foreseeable future. Rather than trying to convince myself to do this ad hoc on my own, working with a trainer ensures this outcome by design. It’s a classic form of the well-known “brain hack” where you force yourself into doing something you know is right for you by creating the conditions in advance, when you’re rationally detached, rather than in the moment, when your baser instincts are in firmer control. And since I pay for my sessions in chunks of 10, and my trainer books our appointments every week, it really keeps me “on the treadmill” (both literally and figuratively).

Of course, there’s more to the value of working with a trainer than just the “forced accountability” to actually show up. My trainer knows what I’m capable of, where my weak spots are, and what my goals are (I just want to be in decent shape, not a body builder or top athlete), and she tailors our workout each time to keep me working on the right exercises with the right intensity and variety. She definitely pushes me harder than I would work on my own (“you can do one last set, can’t you? come on…”), but she also keeps me safe by watching my form and adjusting or eliminating exercises if anything is starting to hurt or look weird. This in turn lets me “turn my brain off” by not having to will myself to keep going or worry about what exercise to do next or whether I’m doing it right (all of which I’d have to do if I were working out solo), and as a result I get a more efficient and impactful workout for the time allotted.

What took you so long?

For several years, “not being in great shape”–or, more specifically, “not having regular exercise be a built-in part of my lifestyle”–has been at the top of my list of personal problems I’d like to improve. I’ve been quite fortunate in terms of family and career (my big problem being “I just want to work all the time”), but I just couldn’t seem to get “work out regularly” checked off my life’s to-do list. Reflecting on why it was so hard for me to tackle this problem–even though I cared about fixing it and had the means–I think it boiled down to the following combination of factors, which many people in my situation also face:

  • It’s not a crisis (yet). I was never “dangerously over weight”, just a bit pudgier than I’d like, and with the situation slowly worsening over time. I gained on average about 5 lbs per year since getting married, though my weight often fluctuated that much in the space of a week, so it was the long-term trend that was a problem, but in the short term it was hard to convince myself “you’ve got to drop everything and deal with this now“.
  • I should be able to fix this on my own. As I mentioned above, I tend to think “I’m a smart, capable person, so I don’t need help from other so-called professionals” when it comes to solving my problems. Especially for getting in shape, I know how to do it, and my needs are fairly modest and normal, so I often told myself that getting a personal trainer amounted to “being a wuss” and admitting a lack of will power or discipline or knowledge, which was “unbecoming for someone of my abilities”.
  • It’s expensive. While I’m well-employed enough to reasonably afford to pay a personal trainer, it’s still a lot of money. My trainer is part of Google’s “g-fit” program, and I pay a subsidized rate of $50/hour (it’s often higher at private gyms). Meeting twice a week works out to roughly $5,000/year for this service. That’s a serious chunk of change, and although “it’s hard to put a price on your health and happiness”, it was enough to further fuel my apprehension about “taking the plunge”.
  • I can’t afford the time. Probably the single biggest deterrent to getting in shape was my inability to convince myself that the few hours a week I would need to dedicate would actually be worth the opportunity cost of less time working or less time spent with my family. As a “startup guy”, both at Plaxo and more recently building Google+, I always feel like there’s more to do and it really matters and the impact is real. In other words, I have the “first-world problem” that I love my job, I’m good at it, and I have the chance to really change the world and get well compensated in the process. So it’s always hard to pull myself away from that, even to spend time with my family, whom I adore and already don’t see enough. Thus taking even more time away to work something as “vain and optional” as improving my physique was just a hard thing to convince myself was a worthy use of my limited time.

Now rationally, of course, I realize that none of these arguments actually make much sense. Of course staying healthy is important and worthwhile, of course it’s a reasonable (and ultimately modest) investment of time and money, and of course “Parkinson’s law” will ensure that my career will not go up in flames because I spend a few hours a week working out. I know of plenty of people who are a lot busier and more important than me that work out regularly and swear by it, so clearly this is a solvable problem with plenty of “existence proofs”. But it held me up for years nonetheless, so I bring it up in the hopes that if some of these excuses ring a bell for you, this might help you fight back.

Is it really worth the investment of money and time?

Obviously not everyone can afford a personal trainer, but I would argue that for most busy professionals, it’s both achievable and money well spent. Most people I know in Silicon Valley work really hard and wish they were in better shape, but are actually bounded more by time than by money to do something about it. So once you convince yourself that you can afford the time, the money should actually be “the easy part”. If someone offered to wave a magic wand and make you healthy and active for the cost of $5,000, I bet a lot of people would take it. Yet that’s exactly what you can do by getting a trainer (and even cheaper if you do small group sessions or only meet once a week). It’s particularly convenient at Google because they have on-site trainers and on-site gyms with lockers/showers (so it really is as simple as “going to my 3pm meeting on the first floor”), but many people live or work near gyms and have flexible enough schedules that they can pencil in time with a trainer a couple times a week. And of course, as many people told me before (and I’m telling you now), it does come with a “high ROI” in terms of having more energy, sleeping better, and spending less time worrying that you’re not getting in shape.

In short, I couldn’t be happier with the choice to start working with a personal trainer, and if you’re anything like me, I bet you’d feel the same if you gave it a try. Let me know if you do (or have already done so recently); I’d love to hear from more people who are figuring out how to be healthy and geeky at the same time!

HipChat is consumer-meets-enterprise done right — check it out!

Three of Plaxo’s best engineers and designers left almost a year ago to start a new company (much as they’d done a few years ago with HipCal, which Plaxo acquired in 2006). After a brief private beta, today they are launching to the public.

HipChatMeet HipChat. It’s a new (and, IMO, very clever and promising) approach to group collaboration within companies and teams–essentially group chat plus file-sharing done with the simplicity and polish of a great consumer app, but targeted at the enterprise. And it’s meant to spread organically and bottoms-up by attracting enthusiastic team members who really find it useful, rather than top-down through long sales-cycles to CIOs–in other words, winning by actually being better for the people that use it every day. You’ll be able to tell this from the moment you start using it–it’s distinctly “un-enterprise-y” in all the right ways, yet every enterprise needs something like this to be more productive and organized.

[ More HipChat screenshots ]

I’m excited about HipChat for several reasons:

First, the founders (Pete Curley, Garret Heaton, and Chris Rivers) are all rockstar talents and super nice guys; the best of the young web 2.0 “bootstrap from nothing and build something genuinely good that grows because people are using and loving it” approach that’s only become feasible recently. Whatever they work on, I know it’ll be well thought through and well executed, and it’ll keep getting better over time. These are good guys to know and watch, and they’re just getting started.

Second, group collaboration is a space that everyone knows is important, and yet that nothing does a good job of solving today. At Plaxo we’ve tried tons of wikis, internal blogs, mailing lists, document depots, dashboards, you name it. They’re always too complicated and cumbersome and never have streamlined workflows that work the way you need. One of my early surprises coming to Google is that for all their efforts and internal tools, the situation is ultimately not much better. Information is still spread everywhere across a variety of systems, is too hard to find and curate, and too often forces you to just ask the person next to you and hope for the best. Maybe new tools like Google Wave will make a difference here, but of course the more flexible and general-purpose a tool like that is, the greater the risk that it will do too many things and none of them just the way you want. HipChat may not be the magic solution to this complex problem either, but it’s refreshing to see the team apply a consumer-app eye and discipline to the problem–focusing on specific task arcs to really nail, and an end-to-end polish and friendliness that’s so clearly lacking from most other groupware tools.

This last point deserves its own slot: in my experience, the only way to really advance the state of technology making a real difference in the lives of real people is to subject it to the harsh Darwinian landscape of consumer software and devices, where if it doesn’t “just work” and provide a compelling and enjoyable experience, it doesn’t get used. This is the sharpening steel that’s honed all the best apps we have today, from gmail to facebook to the iPhone to boxee, and so on. And if you think about it, it’s the missing piece that makes most enterprise software so terrible–your company buys it, and you’re stuck with it, like it or not. The typical enterprise “fitness function” yields a much slower and sloppier rate of evolution than the consumer one, and that I believe is the main reason the quality of the two classes of apps differs so much. So it’s great to see an increase in companies willing to try and swim upstream to gain corporate adoption with a consumer mindset, whether it’s Google Apps,, Yammer, or now HipChat.

If you work on a team, if you’re dissatisfied with the state of collaboration tools, or if you just want to see a really well done new app, I encourage you to check out HipChat. We used several early betas inside Plaxo, and while any new communications tool faces an uphill battle to gain critical mass of adoption and change old habits, enough of us had enough “eureka moments” using HipChat to see its strong potential and to wish that we could fast-forward time until more people are using it and it’s had even more love put into it. The next best thing we can do is to spread the good word and give our support to the team, so consider this post a down payment!

Sources of inspiration for 2010

Despite all the serious challenges the world is facing these days, I’m also seeing more and more that inspires me–cases where great things are happening to great people doing great work and improving the world in the process. Specifically, the following recent examples come to mind:

  • Movies: Avatar
  • Music: Lady Gaga
  • TV: Netflix HD streaming to Blu-Ray/TiVo/Xbox/etc.
  • Mobile: Palm Pre & Nexus One
  • Desktop: Chrome OS
  • Social: Foursquare

What do all of these have in common? They’re all cases of insanely talented outsiders changing the world by just working really hard and doing great stuff.

Everyone said James Cameron was crazy to make Avatar, just like they said he was crazy to make Titanic. They’re both such impossibly grand, expensive, and difficult visions to capture. But he did it anyway, and the work is brilliant, and the only thing that went broke were the previous box-office records.

Same thing with Lady Gaga: just two years ago, no one had heard of her, and she was just playing little clubs in New York. But using her incredible talent in song-writing and performance art, and a willingness to work insanely hard every day, she unleashed her strange and unique vision of music/fashion/art/performance and took the world by storm, becoming the first artist ever to score four #1 hits off her debut album, and at age 23 no less. If you haven’t paid close attention and think she’s just another made-to-order corporate pop starlet, take a closer look, you’ll be surprised (as I was).

Netflix is certainly an outsider when it comes to watching TV in your living room (it’s neither a cable provider nor a set-top box manufacturer), yet now that I can watch entire seasons of Lost in HD on my TV whenever I want–thanks to its Watch Instantly streaming and integrations with existing set-top boxes and gaming consoles–I find myself rarely watching “real TV” any more. And Netflix’s user experience is far superior–it knows which episodes I’ve already seen, so I can just pick up where I left off whenever I have a free moment. And if I’m up at Tahoe for the weekend, I can watch it there too on my laptop, and my episode history is kept in sync because it’s stored in the cloud. Brilliant.

Both Palm Pre and Android (hard to pick a favorite yet!) are up against fierce competition from Apple and the old-world mobile establishment, and neither company (Palm or Google) is an established player in this space, but they’re both producing excellent devices that simultaneously improve the quality of the experience for users while opening up more flexibility and power to developers. And they’re also making web development more of a “first-class citizen” for mobile apps. It’s hard to think of a more ambitious challenge than building your own mobile platform–hardware, OS, software stack, apps, and distribution in physical stores–but that’s not stopping these guys from having a major impact, and the game is just beginning.

Chrome OS isn’t even out yet really, but it’s already clear that the desktop will soon evolve to their vision where all important data lives in the cloud, and it will no longer matter if your computer dies or if you want to use multiple computers in different places–a pain that I’ve experienced many times, as I’m sure you have. Windows is one of the most well-established monopolies there is, so again it’s crazy in some sense to try and compete there, let alone with a radical new vision that does a lot *less* than the status quo, and instead re-imagines the problem in a new way. And yet people buy new computers all the time, so it’s not hard to believe that they could establish considerable market share in a short number of years, while forcing radical change from their competitors at the same time.

And perhaps closest to my own area of work on the Social Web, I think it’s noteworthy that the company that has had the biggest positive impact on how I connect and share with my friends in the last year is not any of the big established players, but a tiny startup that’s building itself up from scratch by making it easier and more rewarding to share where you are and what you’re doing: Foursquare. While I cringe at the amount of work they have to do to integrate with each separate social network and build apps for each separate mobile device (that’s why we need more common standards, of course!), they’re still able to deliver an awesome product with a tiny team, and their service is taking off like a rocket.

Why are these examples so inspiring to me? They provide reassurance that in 2010, two basic things are still true–perhaps more true than ever before:

  1. You *can* win by being excellent and working hard to build a better product
  2. You *can* win even if you’re an outsider in a field of powerful incumbents

It’s hard to believe in transformational innovation if you can’t believe in those two points, and it’s often easy to get discouraged, since these are such difficult challenges. But if those guys can all do it, so can we. In fact, it’s not hard to believe that it’s actually getting easier to succeed in these ways. After all, barriers to entry keep getting lowered, and the spread of information keeps getting faster and more efficient, so the good stuff should be able to be discovered and bubble to the top faster than ever before. If that’s true, then the new “hard problem” should be doing great work in the first place, and that’s the problem I want to be tackling!

What examples are inspiring you right now?

Joseph Smarr has new work info…

High on my to-do list for 2010 will be to update my contact info in Plaxo, because I’ll be starting a new job in late January. After nearly 8 amazing years at Plaxo, I’m joining Google to help drive a new company-wide focus on the future of the Social Web. I’m incredibly excited about this unique opportunity to turbo-charge my passionate pursuit of a Social Web that is more open, interoperable, decentralized, and firmly in the control of users.

I’ve worked closely with Google as a partner in opening up the social web for several years, and they’ve always impressed me with their speed and quality of execution, and more importantly, their unwavering commitment to do what’s right for users and for the health of the web at large. Google has made a habit of investing heavily and openly in areas important to the evolution of the web—think Chrome, Android, HTML5, SPDY, PublicDNS, etc. Getting the future of the Social Web right—including identity, privacy, data portability, messaging, real-time data, and a distributed social graph—is just as important, and the industry is at a critical phase where the next few years may well determine the platform we live with for decades to come. So when Google approached me recently to help coordinate and accelerate their innovation in this area, I could tell by their ideas and enthusiasm that this was an opportunity I couldn’t afford to pass up.

Now, anyone who knows me should immediately realize two things about this decision—first, it in no way reflects a lack of love or confidence from me in Plaxo, and second, I wouldn’t have taken this position if I hadn’t convinced myself that I could have the greatest possible impact at Google. For those that don’t know me as well personally, let me briefly elaborate on both points:

I joined Plaxo back in March of 2002 as their first non-founder employee, before they had even raised their first round of investment. I hadn’t yet finished my Bachelor’s Degree at Stanford, and I’d already been accepted into a research-intensive co-terminal Masters program there, but I was captivated by Plaxo’s founders and their ideas, and I knew I wanted to be a part of their core team. So I spent the first 15 months doing essentially two more-than-full-time jobs simultaneously (and pretty much nothing else). Since that time, I’ve done a lot of different things for Plaxo—from web development to natural language processing to stats collection and analysis to platform architecture, and most recently, serving as Plaxo’s Chief Technology Officer. Along the way, I’ve had to deal with hiring, firing, growth, lack of growth, good press, bad press, partnerships with companies large and small, acquisitions—both as the acquirer and the acquiree—and rapidly changing market conditions (think about it: we started Plaxo before users had ever heard of flickr, LinkedIn, friendster, Gmail, Facebook, Xobni , Twitter, the iPhone, or any number of other companies, services, and products that radically altered what it means to “stay in touch with the people you know and care about across all the tools and services that you and they use”). When I joined Plaxo, there were four of us. Now we have over 60 employees, and that’s not counting our many alumni. All of this is to make the following plain: Plaxo has been my life, my identity, my passion, and my family for longer than I’ve known my wife, longer than I was at Stanford, and longer than I’ve done just about anything before. Even at a year-and-a-half since our acquisition by Comcast, Plaxo has the same magic and mojo that’s made it a joy and an honor to work for all these years. And with our current team and strategic focus, 2010 promises to be one of the best years yet. So I hope this makes it clear that I was not looking to leave Plaxo anytime soon, and that the decision to do so is one that I did not make lightly.

Of all the things I’ve done at Plaxo over the years, my focus on opening up the Social Web over the past 3+ years is the work I’m proudest of, and the work that I think has had the biggest positive impact—both for Plaxo and the web itself. Actually, it really started way back in 2004, when I first read about FOAF and wrote a paper about its challenges from Plaxo’s perspective, for which I was then selected to speak at my first industry conference, the FOAF Workshop in Galway, Ireland. Since that time, I realized what a special community of people there were that cared about these issues in a web-wide way, and I tried to participate on the side and in my free time whenever possible. After leading Plaxo’s web development team to build a rich and complex new AJAX address book and calendar (something that also reinforced to me the value of community participation and public speaking, albeit on the topic of high-performance JavaScript), I knew I wanted to work on the Social Web full-time, and luckily it coincided perfectly with Plaxo’s realization that fulfilling our mission required focusing on more than just Outlook, webmail, and IM as important sources of “people data”. So we crafted a new role for me as Chief Platform Architect, and off I went, turning Plaxo into the first large-scale OpenID Relying Party, the first live OpenSocial container, co-creator of the Portable Contacts spec, co-creator and first successful deployment of hybrid onboarding combining OpenID and OAuth, and so on. Along the way I co-authored the Bill of Rights for Users of the Social Web, coined the term Open Stack, was elected to the Boards of both the OpenID Foundation and OpenSocial Foundation, and worked closely with members of the grass-roots community as well as with people at Google, Yahoo, Microsoft, AOL, Facebook, MySpace, Twitter, LinkedIn, Netflix, The New York Times, and others, often as a launch partner or early adopter of their respective forays into supporting these same open standards. And collectively, I think it’s fair to say that our efforts greatly accelerated the arrival, quality, and ubiquity of a Social Web ecosystem that has the potential to be open, decentralized, and interoperable, and that may define the next wave of innovation in this space, much as the birth of the web itself did nearly 20 years ago.

But we’re not done yet. Not by a long shot. And the future is never certain.

At the recent OpenID Summit hosted by Yahoo!, I gave a talk in which I outlined the current technical and user-experience challenges standing in the way of OpenID becoming truly successful and a “no-brainer” for any service large or small to implement. Despite all the progress that we’ve made over the past few years, and that I’ve proudly contributed to myself, there is no shortage of important challenges left to meet before we can reach our aspirations for the Social Web. There is also no shortage of people committed to “fighting the good fight”, but as with any investment for the future with a return that will be widely shared, most people and companies are forced to make tough trade-offs about whether to focus on what already works today or what may work better tomorrow. There are a lot of good people in a lot of places working on the future of the Social Web, and we need them all and more. But in my experience, Google is unmatched in its commitment to doing what’s right for the future of the web and its willingness to think long-term. One need only look at the current crop of Social Web “building blocks” being actively worked on and deployed by Google—including OpenID, OAuth, Portable Contacts, OpenSocial, PubSubHubbub, Webfinger, Salmon, and more—to see how serious they are. And yet they came to me because they want to turn up the intensity and focus and coordination and boldness even more.

I talked to a lot of Googlers before deciding to join, and from the top to the bottom they really impressed me with how genuinely they believe in this cause that I’m so passionate about, and how strong a mandate I feel throughout the company to do something great here. I also heard over and over how surprisingly easy it still is to get things built and shipped — both new products, tools, and specs, as well as integrating functionality into Google’s existing services. And, of course, there are so many brilliant and talented people at Google, and so much infrastructure to build on, that I know I’ll have more opportunity to learn and have an impact than I could ever hope to do anywhere else. So while there are other companies large and small (or perhaps not yet in existence) where I could also have some form of positive impact on the future of the Social Web, after looking closely at my options and doing some serious soul searching, I feel confident that Google is the right place for me, and now is the right time.

Let me end by sincerely thanking everyone that has supported me and worked with me not just during this transition process but throughout my career. I consider myself incredibly fortunate to be surrounded by so many amazing people that genuinely want to have a positive impact on the world and want to empower me to do the best that I can to contribute, even it means doing so from inside (or outside) a different company. It’s never easy to make big decisions involving lots of factors and rapidly changing conditions, let alone one with such deep personal and professional relationships at its core. Yet everyone has treated me with such respect, honesty, and good faith, that it fills me with a deep sense of gratitude, and reminds me why I so love living and working in Silicon Valley.

2010 will be an exciting and tumultuous year for the Social Web, and so will it be for me personally. Wish us both luck, and here’s to the great opportunities that lie ahead!

Missing the Oscars Finale: A Case Study in Technology Failure (and Opportunity)

Yesterday one of my wife’s friends came over to visit, and we decided on a lark to watch the Oscars (which we haven’t done most years). Even though we pay for Cable and are avid TiVo users, due to a variety of circumstances we missed both the beginning of the Oscars and–more importantly–the entire finale, from best female actress through best picture. My frustration and indignation led to me to think systematically about the various ways that technology could and should have helped us avoid this problem. I decided to share my thoughts in the hope that better understanding technology’s current limitations will help inspire and illuminate the way to improving them. As usual, I welcome your feedback, comments, and additional thoughts on this topic.

The essence of the failure was this: the Oscars was content that we wanted to watch, we were entitled to watch, but were ultimately unable to watch. But specifically, here’s what went wrong that could and should have done better:

  • Nothing alerted me that the Oscars was even on that day, nor did it prompt me to record it. I happened to return home early that day from the Plaxo ski trip, but might well have otherwise missed it completely. This is ridiculous given that the Oscars is a big cultural event in America, and that lots of people were planning to watch and record it. That “wisdom of the crowds” should have caused TiVo or someone to send me an email or otherwise prompt me to ask “Lots of people are planning to watch the Oscars–should I TiVo that for you?”
  • As a result of not having scheduled the Oscars to record in advance, when we turned on the TV it turned out that the red carpet pre-show had started 15 minutes ago. Sadly, there was no way to go back and watch the 15 minutes we had missed. Normally TiVo buffers the last 30 minutes of live TV, but when you change channels, it wipes out the buffer, and in this case we were not already on the channel where the Oscars were being recorded. Yet clearly this content could and should be easily accessible, especially when it just happened–you could imagine a set of servers in the cloud buffering the last 30 minutes of each channel, and then providing a similar TiVo-like near-past rewind feature no matter which channel you happen to change to (this would be a lot easier than full on-demand since the last 30 minutes of all channels is a tiny subset of the total content on TV).
  • Once we started watching TV and looked at the schedule, we told TiVo to record the Oscars, but elected to skip the subsequent Barbara Walters interview or whatever was scheduled to follow. Part way through watching in near-real-time, my wife and her friend decided to take a break and do something else (a luxury normally afforded by TiVo). When they came back to finish watching, we discovered to our horror that the Oscars had run 30+ minutes longer than scheduled, and thus we had missed the entire finale. We hadn’t scheduled anything to record after the Oscars, so TiVo in theory could have easily recorded this extra material, but we hadn’t told it to do so, and it didn’t know the program had run long, and its subsequent 30-minute buffer had passed over the finale hours ago, so we were sunk. There are multiple failures at work here:
  1. TiVo didn’t know that the Oscars would run long or that it was running long. My intent as a user was “record the Oscars in its entirety” but what actually happens is TiVo looks at the (static and always-out-of-date) program guide data and says “ok, I’ll record channel 123 from 5:30pm until 8:30pm and hope that does the trick”. Ideally TiVo should get updated program guide data on-the-fly when a program runs long, or else it should be able to detect that the current program has not yet ended and adjust its recording time appropriately. In absence of those capabilities, TiVo has a somewhat hackish solution of knowing which programs are “live broadcasts” and asking you “do you want to append extra recording time just in case?” when you go to record the show. We would have been saved if we’d chosen to do so, but that brings me to…
  2. We had no information that the Oscars was likely to run long. Actually, that’s not entirely true. Once we discovered our error, my wife’s friend remarked, “oh yeah, the Oscars always runs long”. Well, in that case, there should be ample historical data of the expected chance that a repeated live event like the Oscars should have extra time appended to the recording, and TiVo should be able to present that data to help its users make a more informed choice about whether to add additional recording time. If failure #1 was addressed, this whole issue would me moot, but in the interim, if TiVo is going to pass the buck to its users to decide when to add recording time, it should at least gather enough information to help the user make an informed choice.
  3. We weren’t able to go back and watch the TV we had missed, even though nothing else was being recorded during that time. Even though we hadn’t specifically told TiVo to record past the scheduled end of the Oscars, we also hadn’t told it to record anything else. So it was just sitting there, on the channel we wanted to record, doing nothing. Well, actually it was buffering more content, but only for 30 minutes, and only until it changed channels a few hours later to record some other pre-scheduled show. With hard drives as cheap as they are today, there’s no reason TiVo couldn’t have kept recording that same channel until it was asked to change channels. You could easily imagine an automatic overrun-prevention scheme where TiVo keeps recording say an extra hour after each scheduled show (unless it’s asked to change channels in the interim) and holds that in a separate, low-priority storage area (like the suggestions it records when extra space is free) that’s available to be retrieved at the end of a show (“The scheduled portion of this show has now ended, but would you like to keep watching?”), provided you watch that show soon after it was first recorded. In this case, it was only a few hours after the scheduled show had ended, so TiVo certainly would have had the room and ability to do this for us.
  • Dismayed at our failure to properly record the Oscars finale, we hoped that online content delivery had matured to the point where we could just watch the part we had missed online. After all, this is a major event on a broadcast channel whose main point is to draw attention to the movie industry, so if there were ever TV content whose owners should be unconflicted about maximizing viewership in any form, this should be it. But again, here we failed. First of all, there was no way to go online without seeing all the results, thus ruining the suspense we were hoping to experience. One could easily imagine first asking users if they had seen the Oscars, and having a separate experience for those wanting to watch it for the first time vs. those merely wanting a summary or re-cap. But even despite that setback, there was no way to watch the finale online in its entirety. The official Oscars website did have full video of the acceptance speeches, which was certainly better than nothing, but we still missed the introductions, the buildup, and so on. It blows my mind that you still can’t go online and just watch the raw feed, even of a major event on a broadcast channel like ABC, even when the event happened just a few hours ago. In this case it seems hard to believe that the hold-up would be a question of whether the viewer is entitled to view this content (compared to, say, some HBO special or a feature-length movie), but even if it were, my cable company knows that I pay to receive ABC, and presumably has this information available digitally somewhere. Clips are nice, but ABC must have thought the Oscars show was worth watching in its entirety (since it broadcast the whole thing), so there should be some way to watch it that way online, especially soon after it aired (again, this is a simpler problem than archiving all historical TV footage for on-demand viewing). Of course, there is one answer here: I’m sure I could have found the full Oscars recording on bittorrent and downloaded it. How sad (if not unexpected) that the “pirates” are the closest ones to delivering the user experience that the content owners themselves should be striving for!

Finally, aside from just enabling me to passively consume this content I wanted, I couldn’t help but notice a lot of missed opportunity to make watching the Oscars a more compelling, consuming, and social experience. For instance, I had very little context about the films and nominees–which were expected to win? Who had won or been nominated before? Which were my friends’ favorites? In some cases, I didn’t even know which actors had been in which films, or how well those films had done (both in the box office and with critics). An online companion to the Oscars could have provided all of this information, and thus drawn me in much more deeply. And with a social dimension (virtually watching along with my friends and seeing their predictions and reactions), it could have been very compelling indeed. If such information was available Online, the broadcast certainly didn’t try to drive any attention there (not even a quick banner under the show saying “Find out more about all the nominees at”, at least not that I saw). And my guess is that whatever was online wasn’t nearly interactive enough or real-time enough for the kind of “follow along with the show and learn more as it unfolds” type of experience I’m imagining. And even if such an experience were built, my guess is it would only be real-time with the live broadcast. But there’s no reason it couldn’t prompt you to click when you’ve seen each award being presented (even if watching later on your TiVo) and only then revealing the details of how your pick compared to your friends and so on.

So in conclusion, I think there’s so much opportunity here to make TV content easier and more reliable to consume, and there’s even more opportunity to make it more interactive and social. I know I’m not the first person to realize this, but it still amazes me when you really think in detail about the current state of the art how many failures and opportunities there are right in front of us. As anyone who knows me is painfully aware, I’m a huge TiVo fan, but in this case TiVo let me down. It wasn’t entirely TiVo’s fault, of course, but the net result is the same. And in terms of making the TV experience more interactive and social, it seems like the first step is to get a bunch of smart Silicon Valley types embedded in the large cable companies, especially the ones that aren’t scared of the internet. Well, personally I’m feeling pretty good about that one. 😉

OpenIDDevCamp was hack-tastic!

DSCN5744I spent the weekend in SF at OpenIDDevCamp, hosted at SixApart’s offices. In the style of BarCamp and iPhoneDevCamp, the idea was just to get a lot of people together who were interested in OpenID, provide the space and amenities for them to work together, and let them loose. About 30-40 people showed up, including Brad Fitzpatrick, Scott Kveton, Tantek (with blinking wheels on his shoes), Chris Messina, David Recordon, Christopher Allen, John Bradley, Luke Shepard, and many more.

DSCN5748Over the course of the weekend, I got OpenID 2.0 relying party support deployed on Plaxo, and we found and fixed a bunch of little bugs along the way. You can now use directed identity (e.g. just type in “” as your OpenID and sign in on their side), and you can even use iNames (e.g. I can now sign in with =joseph.smarr). Thanks again to my hacker friend Michael Krelin, who did most of the hard work, and to John Bradley of ooTao for helping me figure out the subtleties of making iNames work properly. David Recordon and I also developed a firm spec for combining OpenID and OAuth into a single OP round-trip–it turns out it’s easier than we thought/feared; write-up to follow shortly. And Chris, David, and I came to a clear consensus on best practices for “remember me on this computer” behavior for OpenID RPs, which I’ll try to write up soon as well.

DSCN5747There was also a lot of great discussion about the future of OpenID, OAuth, microformats, and related technologies, as well as some lively debate on data portability (as you might expect). A personal highlight for me was when Christopher Allen (a co-inventor of SSL) regaled us with tales of how crazy and uncertain the process was to get Microsoft, Netscape, and the other big players at the time to all agree on a common set of principles that laid the groundwork for the development of SSL, which we now take for granted as an essential open web standard. It felt a lot like what’s going on right now in the social web, and the outcome there is an inspirational example to follow.

I’ve said it before and I’ll say it again–I love living and working in Silicon Valley with so many smart, energetic, passionate, and fundamentally nice and optimistic people. To wit: I just gave up a perfectly good weekend so I could stay up past midnight writing code and learning the finer points of XRI resolution, and it felt great! :)

PS: If you eat at Brickhouse cafe, I recommend the “half ass burger”–it’s just the right size. 😉

Tim Berners-Lee groks the social graph

I guess I missed it with the Thanksgiving break, but Tim Berners-Lee recently wrote a thoughtful and compelling essay on the Social Graph, and how it fits in to the evolution of the net and the web so far. Definitely worth a read!

I was pleasantly surprised to see that he references the Bill of Rights for Users of the Social Web that I co-authored, as evidence of recent “cries from the heart for my friendship, that relationship to another person, to transcend documents and sites.” He echoes “the frustration that, when you join a photo site or a movie site or a travel site, you name it, you have to tell it who your friends are all over again,” and points out that “The separate web sites, separate documents, are in fact about the same thing–but the system doesn’t know it.”

I can’t think of a more eloquent way to describe the work I’m doing at Plaxo these days on opening up the social web and enabling true friends-list portability, and it’s certainly inspiring to hear it placed in the context of the larger vision for the Semantic Web by someone who’s already had such an impact on fundamentally improving the ways computers and people can interact online.

Onward, ho!

Older posts

© 2016 Joseph Smarr

Theme by Anders NorenUp ↑