« Page 9 of 18 »

  1. The latest version of JetS3t (jet-set), my Java library for Amazon's Simple Storage Service, is now publicly available.

    Visit the JetS3t web site to download version 0.7.0, to run the latest online applications, or to read the updated documentation.

    Version 0.7.0 includes a number of bug fixes and support for some of the newest features of Amazon S3.

    New functionality:

    • Support for the Amazon CloudFront service
    • Support for configuring and accessing Requester Pays buckets.

    Application updates:

    • Cockpit: Manage your CloudFront distributions, access and configure Requester Pays buckets, easily use DevPay S3 accounts, and have Cockpit remember your preferences
    • Synchronize: More options for setting your configuration options, and improved support for HTTP Proxies

    Other improvements:

    • Improved toolkit support for DevPay S3 account credentials, and for HTTP Proxies
    • .jetset-ignore files now accept wildcard directory paths

    You can also read the full list of changes.

    My thanks go to all those who contributed with code patches, bug reports, feature requests, and general feedback.

    There are comments.

  2. In a concerted push to open up its infrastructure web services to a wider audience, Amazon is building a web-based user interface to their services called the AWS Management Console (console.aws.amazon.com).

    The initial version of the console was released earlier this month with support for the Elastic Compute Cloud (EC2) service. At present, the site provides a user-friendly interface for managing all the basic components of your EC2 servers: instance lifecycles, machine images (AMIs), attachable Elastic Block Store volumes, Elastic IPs, and Security Group firewall settings.

    You access the console using your normal Amazon AWS account credentials, so if you already have an Amazon EC2 account you can sign in directly without any fuss.

    Amazon clearly intends great things for this tool. It will "soon" be extended to support additional Amazon services: data storage with S3, content distribution with CloudFront, messaging with SQS, and database querying with SimpleDB. There are also plans to provide some of the complex server and scaling management features that are the ultimate goals of many cloud-computing users:

    Monitoring, Load Balancing and Auto-scaling -- View real-time monitoring of operational metrics within Amazon EC2, configure load balancing and auto-scaling rules through a web-based UI.

    Ecological Vandalism?

    I must admit I am surprised that Amazon decided to build and provide (for free) their own web UI for these services. On the one hand, the access to powerful, easy-to-use management tools will be a boon to both new and existing AWS users and will no doubt encourage many more people to try their offering. On the other hand, there are a number of third-party businesses that already provide similar web management console tools. These businesses may not appreciate Amazon stepping on -- or thoroughly trampling -- their turf.

    My understanding was that Amazon was going to provide the basic underlying services and APIs, on top of which it would encourage third-party developers and companies to build out a thriving ecology of tools, services and business models. It seems to me that one of the major business models made possible by AWS is now under significant threat.

    I guess it will lead to some vigorous competition where companies will need to find the gaps in Amazon's offering and provide customers with the tools and features that are still worth paying for. The RightScale blog published a detailed and thoughtful response along these lines upon the release of Amazon's console.

    Competition is a good thing. I just hope that Amazon's huge advantages in size and inside-knowledge don't render it the ultimate and only victor. I'm sure that Amazon has the same hopes.

    There are comments.

  3. Amazon has announced an improved request signature protocol for authenticating HTTP requests sent to SimpleDB, to Elastic Compute Cloud (EC2), or to the Simple Queue Service (SQS). The new protocol, version 2, fixes a security flaw in the version 1 signature algorithm.

    The security flaw was identified by Colin Percival, who describes it in detail in his article AWS signature version 1 is insecure. As he says:

    If you are making Query (aka REST) requests to Amazon SimpleDB, to Amazon Elastic Compute Cloud (EC2), or to Amazon Simple Queue Service (SQS) over HTTP, and there is any way for an attacker to provide you with data which you use to construct your request, switch to HTTPS or start using AWS signature version 2 now.

    The good news is that your application will not be vulnerable to this problem if it meets any of these conditions:

    • Sends requests securely using HTTPS
    • Does not include any user-provided data in its request parameters
    • Uses the SOAP API interface, instead of the Query (REST) interface

    Regardless of whether or not your application is currently vulnerable, if it uses the Query API interfaces you should start updating your libraries or code immediately to use the version 2 signature because the old version will be phased out by October 2009. While you are thinking about security, Amazon has provided the article Making Secure Requests to Amazon Web Services which contains recommendations for making secure requests to Amazon Web Services for all interfaces.

    If you need to adjust your application code to use version 2 signatures, read on to see some example code.

    In my book Programming Amazon Web Services (O’Reilly 2008) I demonstrate how to build clients that interact with Amazon's Query API interfaces. Let's look at the ruby code for the AWS class that generates version 1 signatures, and see how to adjust it to work with the new version 2 protocol. (The book's sample code is available in Ruby, Python and Java flavours from http://examples.oreilly.com/9780596515812/. Jump to the end of this post to download the modified AWS class file).

    Version 1 Query Signatures

    It was very easy to generate the version 1 signature for AWS Query requests. Give a hash map of request parameters, it was simply a matter of sorting the parameters alphabetically (ignoring case) and merging them into a string before applying the SHA encryption algorithm. Here is the original signature generation code:

    req_desc = parameters.sort {|x,y| x[0].downcase <=> y[0].downcase}.to_s
    signature = generate_signature(req_desc)
    

    Version 2 Query Signatures

    There are six key differences for generating version 2 signatures, most (but not all...) of which are described in Amazon's documentation for authenticating SimpleDB REST requests:

    • The SignatureVersion parameter must have the value 2 (obviously)
    • Parameters are now sorted by "natural byte" ordering, which means you pay attention to the case of the parameter name when sorting
    • Parameter names and values must now be URL encoded. Be careful to ensure that space characters are encoded as %20, not +
    • When parameters are merged into a canonical string, they are now delimited by = and & characters as with a standard URL. For example: Param1=SomeValue&Param2=AnotherValue
    • The request description string to sign must include the HTTP method name (POST, GET etc), the target host name, and the target request path in addition to the canonicalized parameters string
    • Signatures can now be generated with either the original SHA 1 encryption algorithm, or with the stronger SHA 256 algorithm. To indicate which algorithm you used, you must include the new SignatureMethod parameter with a value of HmacSHA1 or HmacSHA256.

    Here is some ruby code that follows these rules and generates a version 2 signature. It also takes advantage of the new support for the SHA 256 encryption algorithm:

    # Use the strongest HMAC algorithm: SHA 256
    parameters['SignatureMethod'] = 'HmacSHA256'
    
    # Sort, and encode parameters into a canonical string.
    sorted_params = parameters.sort {|x,y| x[0] <=> y[0]}
    encoded_params = sorted_params.collect do |p|
      encoded = (CGI::escape(p[0].to_s) +
                 "=" + CGI::escape(p[1].to_s))
      # Ensure spaces are encoded as '%20', not '+'
      encoded.gsub('+', '%20')
    end
    params_string = encoded_params.join("&")
    
    # Generate the request description string
    req_desc =
      method + "\n" +
      uri.host.downcase + "\n" +
      uri.request_uri + "\n" +
      params_string
    
    # Generate the HMAC signature, using the SHA 256 digest
    signature = generate_signature(req_desc, digest='sha256')
    

    For anyone following along with the original code, I also adjusted the generate_signature method to accept an optional digest parameter that specifies the encryption algorithm to use:

    def generate_signature(request_description, digest='sha1')
      raise "aws_access_key is not set" if not @aws_access_key
      raise "aws_secret_key is not set" if not @aws_secret_key
    
      digest_generator = OpenSSL::Digest::Digest.new(digest)
      digest = OpenSSL::HMAC.digest(digest_generator,
                                    @aws_secret_key,
                                    request_description)
      b64_sig = encode_base64(digest)
      return b64_sig
    end
    

    Of course, in addition to the ability to generate version 2 signatures, the AWS class should retain the ability to generate version 1 signatures if the client requests it. Download the final updated AWS.rb class file to see all the changes -- they are mostly in the do_query method.

    There are comments.

  4. A handy new graphical tool for working with the Amazon Simple DB service has been announced on the AWS Blog. SDB Tool is a Firefox plugin for querying and updating your Simple DB database domains.

    The tool is very functional, if a bit rough around the edges. The interface is a bit intimidating and, on my computer at least, it has a nasty habit of resizing my Firefox browser window when I start it up. However, even at this early stage the plugin provides a straight-forward graphical view into your data that will be very valuable for anyone working with data stored in Simple DB.

    Kudos to Bizo, Inc for developing this tool, and releasing it for everyone's benefit.

    SDB Tool Screenshot

    There are comments.

« Page 9 of 18 »