sameer iyengar

3 key themes from the first AWS Developer Conference

December 4th, 2012 by sameer
Respond

Conference or Concert?

Conference or Concert?

Amazon pulled out all the stops to throw their first Amazon Web Services (AWS) developers conference last week.  Here are three key themes I noticed from the talks they scheduled to fill the time between the laser light shows, delicious food and abundant drinks.

1. Design for failure (and simulate failures)

I lost count of how many speakers stressed the need to be in multiple availability zones.  This is easy to do, and is even easier if you start this from the beginning rather than trying to migrate later on.  The last few EC2 outages resulted in downtime for many popular sites and some bad publicity for AWS.  In response, the team is making it clear that machine failures are expected and a redundant architecture is required.

More importantly, regularly testing failure cases avoids surprises when they happen in real life.  Two strategies:

  1. Closely mirror your development environment to your production configuration and test machine failure and upgrades regularly.
  2. Simulate failure in your production systems.  For example, take down all the machines in one availability zone and ensure that everything is still working.  Netflix has a great tool called ChaosMonkey that makes it easy to run “fire drills” like this.

2. Automate (and monitor) everything

The obvious advantage of cloud computing is the ability to scale capacity to meet demand.  The hidden advantage is the ability to do this automatically. For example, Auto Scaling allows you to automatically launch and terminate new EC2 instances based on defined metrics or on a recurring schedule. This creates an architecture that can respond to unexpected user access patterns the same way that a redundant architecture can automatically respond to unexpected machine failures.

Another example of how to utilize swing capacity is for deployment.  Rather than update existing machines, launch a new cluster of machines with the new code and point traffic to them.  Rolling back is as simple as pointing traffic back to the old machines.  If everything is working, tear down the old ones.

This has two implications for monitoring:

  1. Careful performance monitoring is the best weapon in architecture tuning. The challenge is to identify the right configuration for your application. The logistical and financial headache of transitioning to that configuration no longer exists.
  2. Developers can factor in cost as part of the optimization process.  This is especially advantageous to startups that need to manage cashflow.

3. Data, Data, Data

Immediate access to large amounts of processing and storage capability has always appealed to data enthusiasts, but its clear that companies are making serious progress in this area.  For example, Netflix has built a data processing architecture in which S3 is their “data warehouse” and EMR jobs run when they need to interact with the data.

There are many best practices around the extraction and transform stages of a data warehouse pipeline (essentially, save everything to S3 and use EMR).  I’m excited to see more progress in the later two stages – warehousing and analytics.

  1. Warehousing: Treating S3 as a data warehouse is clever but makes on the fly queries difficult.  Amazon’s Redshift aims to fill this gap with a more traditional data warehouse product.
  2. Analytics: This is the area that I’m most excited for given how cumbersome it is to generate customized reports and charts.  Most companies that aren’t using a traditional enterprise solution like MicroStrategy have had to invest considerable energy to roll their own solution.

 

If you want to relive the brainwashing magic, here are some of the highlight sessions:

Category: uncategorizedNo Comments.

Listen but Filter

November 23rd, 2012 by sameer
Respond

When we first started Beautylish, all the support emails came into my inbox. It was a great way to keep a pulse on things that people were having issues with.

Now we’re a few years in and we have a queue and a process in place to handle inbound emails. I hear about larger recurring issues, but not every single one. I spent last week answering support tickets. It was fun. And incredibly educational.  (Regardless of your role, spend some time doing this!)

I often see the Henry Ford mis-quote “If I’d asked my customers what they wanted, they’d have said a faster horse.” used as a reason to ignore what other people think.

Having conviction in your ideas is important — but knowing how to gather and filter data is equally important. The key is getting to the core of the issue. Knowing someone wants to go faster is useful information. Whether its on a horse, a car or a hoverboard — that’s for you to figure out.

Category: uncategorizedNo Comments.

Instagram’s recipe for killing aggregators: empower content creators

November 5th, 2012 by sameer
Respond

<3 Instagram

I’ve got to get something off my chest. I love Instagram. I’ll spend 15 minutes getting my dinner setting just right so I can heavily saturate the colors on my (rapidly cooling) fettucini alfredo and blur out the wine glass in the background. My friends all make fun of me for it. I don’t care.

I download a lot of iPhone apps.  I like to see what people are building. Instagram is one of the few that I actually use regularly.  Is it really worth a billion dollars?  Probably.  I don’t know.  Doesn’t matter.  This post isn’t about that.  It’s about the next wave of content creation.

Average Joe-tography

Instagram makes it easy to create high quality content.  I like taking pictures.  I don’t own an SLR camera.  I don’t know a whole lot about photography.  My Instagrams still come out pretty nice.  They’re not the best.  They never end up on the popular page.  But, I like them.  Sometimes my friends like them.  And that’s enough to keep me using it.

My (very) limited photography knowledge boils down to two key concepts: composition and exposure.  Both easily controlled by Instagram without even understanding their significance.

  • Composition controls the subject.  There are many composition rules Instagram can’t control, but requiring a square crop on photos forces you to trim out the noise.
  • Exposure controls the lighting.  I don’t fully understand shutter speed, aperture and ISO, but Instagram makes that easy to fake with blur, filters and Lux.

[Aside:  This also illustrates why an “Instagram for Video” is not so simple. Video is a much more complex medium – creation is harder and consumption is harder.  Composition is harder because it takes longer to sift through raw footage.  Exposure is harder because you have to deal with movement.  And it adds another dimension – audio.  Even once you’ve solved those problems, video requires more time to consume and is harder to share.]

Showing Off

User activation is easy for Instagram because its useful as a standalone tool. You don’t need to build a social network before it becomes compelling. Just install it and start making all your photos look like they were taken in the 70’s.  I’ve met plenty of people who don’t follow any friends.  (That is, until I take their phone and force them to follow me.  @iyengar.  Do it.)

But we all crave recognition.  Once you’ve crafted an amazing photo of your dinner, you want people to see it.  Instagram leads to very natural sharing – people inherently want to show off what they create.

Rawr

Rawr

Plus, tweeting “At the beach on a Tuesday #nbd” isn’t as satisfying as Instagramming your sexy legs to really rub in in everyone’s face.  And then tweeting the picture.

Fountain of New

The Internet is very hit driven.  Everyone is bombarded by the breakout winners (think Gangnam Style) until they’re sick of them.  That also means novelty commands a premium – if it’s high quality.

Instagram photos are a constant stream of unique content that hasn’t been seen anywhere else.  On a recent road trip, my friend Ryan and I took Instagrams from the exact same spot at the same time.  We ended up with completely unique perspectives on the same location.

Mendocino Coast by @rypan

Mendocino Coast by @rypan

Mendocino Coast by @iyengar

Mendocino Coast by @iyengar

Flickr and YouTube solved the aggregation and hosting problem for user-generated content. Instagram solves the problem of creating that content in the first place.  In the process, it also became a destination for content consumption.  This makes it more valuable and more sustainable than platforms that depend on content from other sources.  It seems like Mark Zuckerburg agrees.  Users were already posting their Instagrams to Facebook, but owning the source gives you full control the golden content goose.

The Nextagram

Instagram’s content creation loop is simple –

(1) enabling high-quality content creation
(2) leads to natural sharing
(3) resulting in a stream of unique content that inspires more creation.

There are many more of these to come.  Polyvore is another great example of this model.  I’m excited about how these tools will make the next wave of content easy to create, high-quality, unique and endless.

Category: uncategorizedNo Comments.

Want it

October 28th, 2010 by sameer
Respond

I like this story because it applies to lots of things:

A man approaches a Zen Master and asks to be shown the path to enlightenment. The Master replies, “Okay, follow me,” stands up, and walks the man to a nearby river and into the water.

Without warning, the Master forces the man’s head under the water and holds it there as he struggles violently for his life, until he is nearly dead.

At last the Master pulls the man up, gasping for air, and says, “When you want to be enlightened as badly as you wanted to take your next breath just now, come back and see me.”

Really wanting something helps you overcome obstacles to get it.

Category: uncategorizedNo Comments.

Cool press for Google TV Ads

June 13th, 2010 by sameer
Respond

In the wake of the Google TV announcement, Fortune had a cool article about Google TV Ads — Google TV = cool. But Google TV Ads = $$$.

TV Ads is even possible because television isn’t so old media anymore. The technology is there to let Google do what it does best: process data. Through information provided by set-top boxes from its partner Dish Network, Google lets TV advertisers target specific channels and shows, and identifies not only how many people saw their ads, but also the demographics of the audience.

And if you missed this a few months ago. Slate did a great video overview.  Its always fun to see stuff you work on in the press!

Category: advertisingNo Comments.

Guilty Pleasures

December 26th, 2009 by sameer
Respond

What you don’t share is just as telling as what you do.  Check out this Last.fm mashup that shows which tracks people most often delete from their profile of scrobbles.

Most deleted scrobbles from last.fm.

Most deleted scrobbles from last.fm.

Conclusion: People don’t want to admit they like Lady Gaga and Britney Spears.

Last.fm also released their Top 40 of 2009 and, more interestingly, released some raw data for people to play around with.  Analytics are fun and I’d love to see more companies make this kind of aggregate data available.

For example, it let me find this fun fact — Of the top 1000 artists on Last.fm, Lady Gaga received a whopping 3% of the 646 million playbacks.  I’m guessing that doesn’t even count the ones that people later decided to delete.

Category: analyticsNo Comments.

@popularnow: Search Trends + News + Twitter

December 6th, 2009 by sameer
Respond

Check out @popularnow on Twitter.

Its a mashup I wrote that tries to find interesting news based on Google Search Trends. It uses Yahoo! BOSS and the Twitter API and runs on Google App Engine.

Few things I learned:

  • App Engine makes writing simple apps and rapid prototyping a breeze.
  • Yahoo’s APIs are incredibly developer-friendly.
  • Twitter spam-bots will find you even before you tell anyone about your account.

So far, it’s found some interesting articles and a whole lot of Tiger Woods gossip.   I guess that’s the sad reality of what people have been searching for over the past week.  Even cooler, it somehow got two retweets.

Follow it and let me know what you think…

Category: programmingNo Comments.

Pair Programming is Awesome

November 9th, 2009 by sameer
Respond

I’ve been meaning to write about this for a while.  Pair programming is awesome.  I was a huge skeptic when it was first pitched to our team, but after trying it a few times, I was completely sold.  It makes you more productive, produces better code and is just more fun all around.

I’m know there has been tons of stuff written about this, but when I read it, I was so skeptical that I thought all those authors were crazy.  But you know I’m not crazy.  So I’m going to tell you why its great.

Pear by hassi on Flickr

Photo from hassi on Flickr.

Benefits of pair programming

Not convinced?  How come?  I already said it was awesome (and I put it in bold).  Here are some benefits –

  1. It keeps you focused. If you’re anything like me, you probably have a tendency to drift when you hit a stopping point.  Maybe check some e-mail or read your favorite blog.  Not anymore.
  2. It helps when you get stuck. Its easy to spend a lot of time trying to figure out something that you’re unfamiliar with.  It always helps to have someone to discuss it with without having to feel bad for interrupting your co-workers all the time.
  3. It avoids stupid mistakes. Have you ever spent 30 frustrating minutes trying to figure out why your change isn’t showing up in the output, only to realize that you forgot to hit save?  Yes, yes, you have.  Your partner is likely to notice those things before you spend unnecessary time pulling your hair out.
  4. It produces better code. It’s less tempting to take shortcuts, write unreadable code and  ”forget” to test things when you’re working with someone.
  5. Its the best way to learn a new code base. The best way to get someone up to speed on a project is to have someone pair program with them.  They’ll spend less time digging around outdated documentation and more time contributing.  I’ve effectively pair programmed with people outside my team who knew absolutely nothing about the project coming in.
  6. Its more fun. This one really depends on the people you’re working with, but its definitely a big upside.

How to set up a pair programming workstation

The set-up is really, really important.  Both people need equal access.  You’ll need:

  • Two keyboards
  • Two mice
  • A long, straight desk (no weird angles)
  • A giant monitor

Its key for each person to have their own keyboard and mouse so that no one is just sitting there passively watching the other person type.

How to pair program

Now that you’ve got a sweet workstation set up, its time to actually do it.  This part is easy, just start writing code.  Switching off can be a bit tricky at first, but soon it’ll become seamless.

If you’re using Test-Driven Development (writing a test before writing a feature), you could have one person write a test that fails, then have the other person make the test pass and write the next test.  Its an easy way to make sure you’re switching off regularly.

One of the big advantages is that its easier for something to do or fix something than to spend time explaining it.  Notice that something is spelled wrong?  Fix it, you have your own keyboard.  Have an idea for how to do something?  Just write it and let the code speak for itself, instead of trying to explain what you’re trying to do.  You can always delete or edit it as necessary.

Potential Downsides

  1. It could take longer than two people working independently. This is true, but I’d argue only if both people know exactly what they’re doing.  More often than not, when you’re working on something new or complex, pair programming will be faster.
  2. You could end up hating your co-worker after spending so much time with them. This is true too.  Definitely switch things up often to avoid this.

Love pair programming? Still not convinced?  Leave a comment.

Category: productivity  programmingNo Comments.

Revenue?

November 7th, 2009 by sameer
Respond

An interesting quote from “The Way I Work“:

Kids are coming out of school thinking, I want to start the next YouTube or Facebook. If a restaurant served more food than everybody else but lost money on every diner, would it be successful? No.
– Jason Fried

Having lots of users wont automatically generate revenue but it does indicate that you’ve got a useful product.  If something is useful, then you can probably make some money from it.  (Though its probably not as easy as slapping some ads on it and waiting for the checks to come in.)

You have to think about what makes the product useful (either to the user or an advertiser or another party).  If you’re looking to run a business, its probably okay to turn away some users in favor of converting the rest to customers.

Category: advertising  entrepreneurshipNo Comments.

Happy

November 7th, 2009 by sameer
Respond

Love this image that I found over at Minimal.  I think it looks even better with the colors inverted:

happy

Category: productivityNo Comments.