you're reading...

Founders BLOG

WebStor – new open source high performance API for Amazon S3

OblakSoft is pleased to announce the release of WebStor 1.0b.1.1– the library providing high-performance cloud storage access.

WebStor is used in ClouSE – the Cloud Storage Engine for MySQL.   ClouSE makes cloud storage to be a drop-in replacement for local storage.   The outmost efficient and reliable cloud storage access is one of the key requirements that make the solution viable.

We’ve got a lot of questions about how ClouSE achieves such a great performance working with cloud storage.  Cloud storage has a perception of being slow, but ClouSE works with cloud storage as fast as with local storage.  WebStor is one of three pillars for high-performance cloud storage access (compression and caching are the other two).

In our benchmarks we were able to achieve up to 80+ MB/s transfer rate with Amazon S3.  While ClouSE needs much, much less than that, thanks to compression and caching, when push comes to shove, WebStor is up for the job.

Here is the WebStor performance benchmark chart for transferring data from / to Amazon S3:


The chart shows how the transfer rate was growing, as the number of parallel operations was increasing from 1 to 64.  It started with ~8 MB/sec with one operation running at a time, and grew all the way to ~80 MB/sec with 64 parallel operations.  The benchmark was run in the Amazon US-standard region.  The object size used in the benchmark was 1 MB.   The benchmark source code is included along with the WebStor library.

WebStor has the following features:

-          Supports Amazon S3 and Eucalyptus Walrus.  Other cloud storage provides may be added in the future.

-          Runs on Linux and Microsoft Windows.  Should be easily portable to Linux-like platforms.

-          Supports parallel put / get / del operations which allow achieving high throughput.

-          Supports SSL and plain-text HTTP.

-          Supports HTTP proxy.

-          Supports Amazon S3 multi-part uploads.

-          Provides C++ API.  APIs for other programming languages may be added in the future.

WebStor source code is available under the Apache 2.0 license absolutely FREE.  Get your own copy now from at http://www.oblaksoft.com/downloads.

Discussion

Trackbacks/Pingbacks

  1. [...] WebStor – new open source high performance API for Amazon S3 … [...]

Post a Comment

Categories

Twitting ...