Amazon S3: Now Much Safer for Important Data

A few weeks ago when I spoke at the St. Louis Cloud Computing User Group, one of the possible cloud storage worries I brought up was the prospect of a few misplaced (accidental or malicious) clicks deleting large swaths of data. This applies with both S3 (the market leader) and other similar offerings. If you’ve tried out the various GUI tools for manipulating S3 “objects”, you’ve no doubt noticed that just a few clicks could delete thousands of objects (files) or even a whole bucket. Imagine a naive new employee (or worse) discarding terabytes of customer data; your business could be flushed down the drain in seconds.

Amazon has recently added a couple of features which greatly reduce this risk: Multi-Factor Authentication and Versioning. Using these features, it is now much more reasonable to store important data on S3 – the access needed to delete data can be controlled in such a way that even a malicious user, with access to credentials sufficient to do real work, nonetheless won’t be able to actually delete any data.

As the various cloud offerings mature, I expect all major providers to offer increased “safety” features, and for technical audits to verify and require their use.

iPad: Yet Another Opinion

Here are my initial, general thoughts about the much-hyped iPad. Clearly the world doesn’t need another blog post about this, but it sets the stage for something coming next.

  • As many have observed, iPad is most easily summarized as a larger iPod Touch, plus some of the mobile data capability of an iPhone. Although this has been expressed widely as a criticism, I note that a very large number of people have bought an iPod Touch or iPhone.
  • By making the iPad fit the above description so well, I fear that there is a tinge of Apple playing it safe for Wall Street. Playing it safe, has not been the strategy that invigorated Apple (and its financial performance) over the last decade.
  • This iPad “1.0” is somewhat short on hardware features. I suspect a second generation device will arrive in 2011 with a few more ports, more storage, more wireless, etc. 1.0 only has to be good enough to prime the market for 2.0.
  • The screen needs more pixels; the resolution / DPI is unimpressive. Also, OLED would have been nice; but Apple had to trade off some things to get to a price point, and the screen technology was obviously one of them.
  • The battery life Apple claims, even if it is vaguely close to reality, is fantastic.
  • I am surprised at the lack of a video camera.
  • I expect to see some kind of trivial tethering interoperation between iPad and iPhone over Bluetooth, sometime in the next couple of revisions of both products. I suspect that loyal Apple fans carrying an iPhone 3GS will end up able to use their iPhone mobile voice/data service for both devices… possibly with some extra monthly service charge.
  • iPad 1.0 will not replace Kindle or other eBook readers, though it might slow their sales growth a bit. But what about iPad 2.0, 3.0, with a better screen and even longer battery life? Once a beautiful color LCD device is good enough, monochrome eInk will be a very tough sell.
  • I will quite likely buy an iPad shortly after it ships; but I’ll be buying perhaps 25% to enjoy it as a consumer, and 75% as a means of more fully understanding the industry importance of the tablet form factor.
  • As a user of a “real” Apple computer (a MacBook Pro running OSX 10.6), I find the closed App Store software distribution model something of a disappointment, compared to a tablet form factor Mac OSX PC I could easily imagine; but I have another blog post coming about that in a few days, after I get some real (non-punditry) work out the door.

Upcoming talk: Cloud Computing User Group

The St. Louis Cloud Computing User Group launches on Jan. 21st at Appistry. Sam Charrington over there kicked it off, but I suspect it will shortly grow far past its Appistry roots.

I’m giving a talk (one of two) at the first meeting. Contrary to the initial description floating around, I won’t be speaking (in detail) about “Amazon Web Services from a Developer Perspective”. Rather, my talk will be broader, and from a developer+business perspective:

To the Cloud(s) and Back

Over the last few years, I’ve been to the Amazon cloud and back; on a real project I started with inhouse file storage, moved to Amazon S3, then moved back. I’ve likewise used EC2 and tried a couple of competitors. I think this qualifies me to raise key questions:

  • Should you use (public) cloud storage? Why and why not?
  • Should you use (public) cloud CPUs? Why and why not?
  • How do you manage an elastic set of servers?
  • Can you trust someone else’s servers? Can you trust your own?
  • Can you trust someone else’s sysadmins? Can you trust your own?
  • What about backups?

This talk will mostly raise the questions, then offer some insights on the some of the answers.

Update: Slides are online here.

Unrealistic Cost Expectations, and How to Fix Them

I suppose there have been hiring companies with wildly unrealistic cost expectations forever; the internet just makes it more visible. Take, for example, this job post for PostgreSQL expert, which I republish here for criticism and comment, anonymized:

We are looking for a postgre expert with indepth Oracle skill to help with the following project:

1) migrate all data and structure from Oracle 9i to PostgreSQL 8.3.
2) create a script to capture daily differentials on Oracle db and export the changes to PostgreSQL
3) create a script to automate the import the Oracle differential export into PostgreSQL on a daily basis
4) full documentation

Will provide both Oracle and Postgre dev box to work with, interested party please send email to (REDACTED)

Job budget between USD 300 to USD 400. However need this delivered within one week from job acceptance, or before Dec. 31, 2009, whichever come first.

To clarify for anyone reading this, this is not my job post. I am not looking for a PG expert. Do not contact me to apply for this work.

This fellow wants:

  • An experienced guru
  • In two quite complex technologies, one of which is a very expensive technology
  • To do a non-trivial project, and presumably, to be responsible for the results actually working
  • Who can do their project Right Now
  • Over the Christmas holiday, at least here in the US
  • For a $400

It seems to me that this person, in addition to creating some annoyance on the mailing list where they posted it, simply has wildly unrealistic expectations. As a result, they are likely to end up disappointed with any real person applying for their work. They will quite likely get multiple applicants, eager to attack the job for the budget shown; so I am not suggesting that noone will do it.

Instead, I estimate that most likely a week will come and go, $400 will come and go, and they will not have a working system. With some struggle and legwork on the hiring end, they may get the end result for a surprisingly small multiple of the proposed budget… but if they started with a more realistic amount in the first place, they’d likely get there faster and with less work on the hiring end.

A broader lesson, that I’ve learned through experience in the trenches, is that if you don’t have a good feel for the price range, start with no price range. Then talk with the first handful of applicants, listening carefully. With a couple of hours (for a simple request), you’ll probably have at least some realistic sense of the size of your project. With this knowledge, you can make more realistic and credible job posts, yielding more and better applicants.

Were you hoping for an approach to fix someone else’s unrealistic expectations? Sorry, I’ve not found a good way to do this. The best you can do is to find and fix your own.