FAQ

If you cannot find the answer to your question, please don’t hesitate to ask us at info@oblaksoft.com.

[ expand | collapse ]

General

  • What is ClouSE?
    ClouSE is the Cloud Storage Engine for MySQL that can utilize cloud storage such as Amazon S3 or Google Cloud Storage to store relational and blob (a.k.a. structured and non-structured) data. ClouSE provides fully functional, transactional and ACID-compliant relational data management on top of cloud storage.

  • What storage providers can my data be stored in with ClouSE?

    ClouSE v1.0 supports Amazon S3 and Google Cloud Storage as public cloud storage utility providers. In Amazon S3 case you can choose any of the Amazon Regions to store your data. Pick a Region to optimize for latency, minimize cost or address regulatory requirements.

    ClouSE also supports Eucalyptus Walrus as a private cloud storage utility provider that can be run on your premises.

  • Is my data stored securely in the cloud?

    ClouSE separates database management from storage management: relational database management (such as query processing, transaction management, etc.) happens on the machine that ClouSE and MySQL run while the cloud provides the storage for the data. To guarantee data confidentiality in the cloud ClouSE can encrypt relational data using strong AES-256 encryption before it leaves the database management site. This ensures that the data is visible neither on the wire nor at the cloud storage provider site. In that respect, ClouSE is the only cloud-based relational data management technology available today that solves the problem of data confidentiality in the cloud.

  • What is unique about ClouSE?

    While ClouSE provides functionality that can be found in other cloud-based solutions ClouSE deserves its own category.

    Unlike Database-as-a-Service (DBaaS) technologies ClouSE:

    • provides full data confidentiality in the cloud,
    • does not require database access only over high-latency and less reliable network, so the applications can enjoy the standard topology where an application server and a database are deployed on the same machine or over fast reliable LAN,
    • provides seamless integration of structured and non-structured cloud data (weblobs),
    • elastically scales storage consumption both up and down,
    • uses cloud storage directly and does not require additional storage to implement data protection (backup or replication).

    That said a DBaaS provider could use ClouSE to power the DBaaS offering to achieve full storage elasticity, save money on storage and offer non-structured cloud data integration to their consumers.

    Unlike non-relational data store technologies (e.g. Amazon SimpleDB, Windows Azure Table Service, CouchDB, etc.), ClouSE:

    • provides full data confidentiality in the cloud,
    • does not require database access only over high-latency and less reliable network, so the applications can enjoy the standard topology where an application server and a database are deployed on the same machine or over fast reliable LAN,
    • fully supports the standard, widely used and well known ACID-compliant relational programming paradigm,
    • supports fast data access through multiple secondary indexes,
    • supports transactional updates spanning multiple tables,
    • supports structured queries spanning multiple tables.

    In addition, ClouSE non-structured cloud data integration (weblobs) may achieve some scalability benefits of non-relational data store without changing the programming paradigm.

    And last but not least, ClouSE is the first cloud-based RDMBS solution that truly does not require existing applications to change in order to take advantage of cheap, reliable and elastic cloud storage.

  • Is cloud storage reliable enough to be the primary data storage?

    Cloud storage is a service available over the Internet that provides highly available highly reliable virtually unlimited data storage where data protection is implemented by storing data redundantly. Here is how Amazon describes their Simple Storage Service (a.k.a. S3): “… Amazon S3 provides a highly durable storage infrastructure designed for mission-critical and primary data storage. Objects are redundantly stored on multiple devices across multiple facilities in an Amazon S3 Region. To help ensure durability, Amazon S3 PUT and COPY operations synchronously store your data across multiple facilities before returning SUCCESS. Once stored, Amazon S3 maintains the durability of your objects by quickly detecting and repairing any lost redundancy. Amazon S3 also regularly verifies the integrity of data stored using checksums. If corruption is detected, it is repaired using redundant data. …”. For more information about Amazon S3 please refer to http://aws.amazon.com/s3/.

  • Is cloud storage a viable choice for relational data?

    It is now, with the help of ClouSE.

    Designed and optimized for cloud storage from ground up ClouSE implements strict consistency enforcement, compression and caching (both read and write) techniques to smooth out cloud storage adoption bumps such as non-strict data consistency, high-latency storage access and higher storage access failure rate due to network failures.

    To guarantee data confidentiality in the cloud ClouSE can encrypt relational data using strong AES-256 encryption before it leaves the database management site. This ensures that the data is visible neither on the wire nor at the cloud storage provider site.

  • Will I save money with ClouSE?

    Yes, in many ways.

    First, ClouSE allows utilizing cloud storage to store relational data. The total cost of ownership (TCO) of the cloud storage is sizably less than a comparable investment in similar grade on-premise storage. The cloud storage utility provider (such as Amazon S3 or Google Cloud Storage) stores data reliably and you pay only for the storage you use. Thus you won’t have to invest into your own high-reliability storage, ongoing management, perform backups or over-provision the storage.

    Second, ClouSE uses compression and caching (both read and write) techniques for relational data to reduce storage and network bandwidth consumption thus saving money on storage and network bandwidth.

    And last but not least, ClouSE provide a simple and elegant programming model for using cloud to scale out content delivery which will help to save money on application development and support.

  • Can ClouSE be used with my private cloud?

    Yes, as long as the private cloud storage provides Amazon S3-compatible REST API. In particular ClouSE supports Eucalyptus Walrus as a cloud storage utility provider.

    In our development and testing we found private cloud storage being very convenient pre-production testing environment prior to deploying data to the cloud.

    Please refer to http://www.oblaksoft.com/docs/clouse.pdf for further documentation on Private Cloud Configuration for ClouSE.

  • What is WebStor?

    WebStor is an open source library developed by OblakSoft that provides highly performant and robust access to cloud storage. ClouSE uses WebStor as a part of cloud storage driver infrastructure to access storage offered by cloud providers such as Amazon S3 and Google Cloud Storage. For more details refer to WebStor documentation.

Getting started with ClouSE

  • Where do I get ClouSE?

    ClouSE v1.0 Beta software for Linux x86 and x64 can be downloaded for FREE from http://www.oblaksoft.com/downloads/ after a quick registration.

  • Do I need to change my MySQL application to store data in the cloud with ClouSE?

    Most MySQL-based applications wouldn’t require changing the application code; the decision to leverage cloud storage for the application data can be made by the DBA as part of application database deployment, much like tablespace management decisions. ClouSE does not require database access only over high-latency and less reliable network, so the applications can enjoy the standard topology where an application server and a database are deployed on the same machine or over fast reliable LAN.

  • How do I migrate my data to the cloud?

    It is easy. Once ClouSE is installed and configured, the ALTER TABLE table_name ENGINE=CLOUSE; MySQL statement can be used to migrate the table_name table to the cloud. And voilà – your data is in the cloud. ClouSE will take care of migrating all existing relational data for these tables to the cloud storage.

  • Do I have to migrate all my databases to cloud at once?

    Not at all. ClouSE supports data migration on the table level, so you can take migration at your pace and migrate your data table by table.

  • Is it possible to migrate data out of ClouSE?

    Yes. The data is can be easily migrated to and from the cloud, in and out ClouSE.

    We hope that you’d never want to do that and we would be glad to hear your feedback at feedback@oblaksoft.com so that we could improve ClouSE to address your issues. However should you make a decision to migrate data out of ClouSE it is just as easy as migrating data into ClouSE (with the only complication being that you’d need to invest again into local storage and data protection).

  • What is the easiest way to try out ClouSE?

    We worked real hard on ensuring the simplicity of the trials. As a convenient starting point OblakSoft provides an EBS-backed AMI with Amazon Linux, MySQL server community edition and ClouSE preinstalled. To finalize configuration, you need to provide cloud storage connection options. For more information about OblakSoft AMI please go to http://www.oblaksoft.com/downloads/. Now you are ready to call ALTER or CREATE TABLE with ENGINE=CLOUSE option and voilà – your data is now in the cloud.

    Another way to try ClouSE is to install Eucalyptus Walrus and configure ClouSE to use it as a cloud storage utility provider in a private cloud scenario. For more information please refer to the “Private Cloud Configuration for ClouSE” in the http://www.oblaksoft.com/docs/clouse.pdf.

ClouSE technology

  • What consistency model should I expect from data managed by ClouSE?

    While the cloud storage by itself may provide only eventual consistency guarantees, ClouSE provides fully functional, transactional and ACID-compliant relational data management on top of cloud storage. Designed and optimized for cloud storage from ground up ClouSE implements strict consistency enforcement, compression and caching (both read and write) techniques to smooth out cloud storage adoption bumps such as non-strict data consistency, high-latency storage access, and higher storage access failure rate due to network failures. As a transactional storage engine for MySQL, ClouSE opens cloud storage to SQL developers and SQL-based applications. ClouSE reduces the need to retrain developers and rewrite applications to use new data management paradigms, thus streamlining cloud storage adoption by lowering the barrier for it.

  • Can I make sure that the data is accessed only by me?

    Yes. The relational data access can be fully controlled on the database management site by using strong AES-256 encryption, so if you run database management on your premises with encryption enabled the ability to access your relational data never leaves your premises. You are responsible for the encryption key management and for safeguarding these keys.

    In that respect, ClouSE is the only cloud-based relational data management technology available today that solves the problem of data confidentiality in the cloud. This overcomes a very important cloud adoption barrier and exposes cloud to workloads that previously wouldn’t be able to take advantage of the cloud for security reasons.

  • How much data can I manage with ClouSE?

    One ClouSE v1.0 instance can manage up to 64 TB of relational data and unlimited amount of weblob data. There is no need to make upfront commitments and the storage consumption scales up and down elastically with the amount of managed data.

  • Do I need to specify an initial storage size for ClouSE to use?

    No. There is no upfront commitment and the storage consumption scales up and down elastically with the amount of managed data.

  • Would storage consumption decrease if less data is stored?

    Yes. Unlike some other cloud-based relational data management solutions, with ClouSE the storage consumption is fully elastic and scales both up and down with the amount of managed data.

  • Does ClouSE optimize data and bandwidth consumption?

    Yes. Designed and optimized for cloud storage from ground up ClouSE uses compression and caching (both read and write) techniques for relational data to reduce storage and network bandwidth consumption.

  • Do you offer high-availability for the data storage?

    Once ClouSE delivers the encrypted user data to the cloud storage, the storage availability is guaranteed by the cloud storage utility provider. ClouSE works with Amazon S3 and Google Cloud Storage that provide highly reliable highly available storage. Availability is backed with the Amazon S3 Service Level Agreement which today is equal at least 99.9% Monthly Uptime Percentage. For further details on Amazon S3 SLA please refer to the corresponding documentation at http://aws.amazon.com/s3-sla/ as well as the 3rd party independent analysis of Amazon S3 services.

    In the case of the private cloud storage utility provider, such as Eucalyptus Walrus, the storage availability and reliability depends on the Eucalyptus Walrus configuration. For further details on Eucalyptus Walrus configuration please refer to http://www.eucalyptus.com. When using Eucalyptus Walrus please ask the personnel responsible for running the Eucalyptus Walrus service for their SLA to make sure they provide the level of reliability and durability acceptable for your data.

  • Where can my database management run?

    Anywhere, as long as it has a connection to the cloud storage.

    You can run database management on your premises or in any hosted environment (such as Amazon EC2, other compute clouds, hosters, etc.) that supports running Linux, MySQL and custom storage engines. You have flexibility of choosing the topology and location for the configuration that best fits your unique requirements to enforce data confidentiality, optimize for latency between the application and the database, minimize cost, address regulatory requirements, etc.

    It is important to note that the best way to ensure your data can be accessed by you and you only is to run database management on your premises. You can generate your own encryption key that will be used to encrypt the data before it leaves your premises. This ensures end-to-end data safety as the access to data is not granted to the cloud storage provider. In that respect, ClouSE is the only cloud-based relational data management technology available today that solves the problem of data confidentiality in the cloud. This overcomes a very important cloud adoption barrier and exposes cloud to workloads that previously wouldn’t be able to take advantage of the cloud for security reasons.

  • What happens if the database management machine is lost?

    The database cache is lost, but the data is SAFE in the cloud. As long as you provide the correct encryption keys it is very easy to set up and configure a new database management machine (physical or virtual) to run ClouSE and recover local data from cloud.

    As ClouSE uses the cloud storage directly, it only needs a small amount of metadata to be present locally in order to resume database management. So with ClouSE the database will get recovered in minutes, not in hours or days like restoring from a traditional RDMBS backup.

  • What happens when I need to migrate database management to a new server?

    It is very easy to set up and configure a new database management machine (physical or virtual) to run ClouSE. Then just copy MySQL configuration and metadata as well as ClouSE files (under a couple GB on most configurations) to the new machine.

  • Do you support SSL connections to the cloud storage?

    Absolutely, this is a default setting for public cloud storage. In conjunction with data encryption on the database management site, this guarantees full confidentiality and integrity of your data in the cloud.

    SSL is not supported for Eucalyptus Walrus (the data still can be encrypted on the database management site). If this is something that would block your adoption of ClouSE please let us know at feedback@oblaksoft.com.

  • What library ClouSE uses to access cloud storage?

    We couldn’t find any existing library meeting all our requirements (performance, reliability, feature set, etc). Thus we have developed our own WebStor library in C++. We made its sources available for public use under Apache 2.0 license. For more details refer to WebStor documentation.

Weblob

  • What is a weblob?

    Weblob is a feature of ClouSE (stands for web-accessible blobs). Each blob may contain non-structured data such as photo and video images, documents, audio files, etc. From the database perspective, weblobs are regular blobs and fit naturally into the ACID model. However they also expose URLs that can be used by the client, typically a web browser, to download the weblob content directly from the cloud storage utility provider, bypassing the application server. Weblobs provide a simple and amazingly powerful model for a relational database developer to leverage the cloud to scale out content delivery while using transaction guarantees to ensure consistency between connected relational and non-relational data. For large classes of applications this will lead to a significant workload scale-out.

  • Why should I use weblobs instead of local files for content delivery?

    There are three reasons why weblobs are better than local files:

    1. Weblobs are stored under the same level of protection as the relational data. Local files are not protected by the relational data protection mechanisms such as database backup and replication and need to have a custom data protection strategy.
    2. Weblobs can leverage the cloud power to scale out content delivery: weblob content is served by the cloud storage utility provider which takes load off the application server. Local files are served by the application server and consume network bandwidth, disk I/O and CPU power of the application server.
    3. Weblob access is ACID-compliant so weblobs can be updated in a transaction along with the associated relational data and this transaction can commit or roll back as a whole. Updates to local files can succeed or fail independently from relational data updates, thus requiring writing intricate custom application logic to properly handle failure conditions.

  • Can I use weblobs to store large content?

    Yes. Weblobs provide streaming functionality that can be used to upload very large content in smaller chunks thus avoiding memory pressure on the server.

  • What is the programming model for weblobs?

    In MySQL a weblob is expressed via a pair of BLOB fields that have a special naming convention: field_name$wblob and field_name$wblob_info. The following example shows how to create the pictures table that is keyed by the id and has a picture weblob:

    CREATE TABLE pictures (id BIGINT KEY, picture$wblob LONGBLOB, picture$wblob_info BLOB) ENGINE=CLOUSE;

    picture$wblob is going to store the actual picture while picture$wblob_info is going to return the URL of the picture in the cloud storage. The URL can be used by the client (e.g. a web browser) to download the picture directly from the cloud storage.

    On update picture$wblob_info can be used to specify streaming commands and/or object name for the weblob URL.

Websites on cloud storage

  • Is there a sample application and/or code available that demonstrates the power of ClouSE?
    Yapixx shows the easiest way to build a dynamic WordPress website on cloud storage. Yapixx along with its source code is available on a fully configured FREE AMI.
    And of course feel free to take any of your existing applications working on top of MySQL and migrate it to the cloud storage. It’s just a matter of executing the ALTER TABLE table_name ENGINE=CLOUSE statement for the application tables.
  • What is YAPIXX?
    Yapixx is a ready-to-run WordPress on cloud storage. Yapixx serves as an example of a beautiful yet powerful dynamic Website that seamlessly gets all the benefits of cloud storage – high availability and reliability, quick and easy disaster recovery, and serving content in a highly scalable fashion. Yapixx is an acronym for Yet Another Picture Sharing Site.
    Yapixx is WordPress, configured as a picture sharing site, that stores all its data in cloud storage (thanks to OblakSoft ClouSE). Yapixx demonstrates how easily website pages as well as its media can be stored and served directly from cloud storage.
    Yapixx also serves as an open source sample of a complete cloud-based web application that was moved to the cloud without changing a line of code to run on top of ClouSE storage engine.
    Yapixx (along with its source code) is available on a fully configured FREE ready-to-run AMI. For more information refer to Yapixx documentation.
    Yapixx works best in the combination with WP2Cloud WordPress plugin.
  • What is WP2Cloud?

    WP2Cloud is a plugin that extends WordPress to store media files (such as pictures) on the cloud storage. Media files that are uploaded to the WordPress are actually uploaded to the cloud storage, not to the local file system of the Web server hosting this website.
    The media files are served from the cloud, the website pages now refer to the media files via direct cloud storage URLs ( weblobs).

    WP2Cloud plugin kills two birds with one stone: (1) files are stored reliably in the cloud and don’t need to be backed up, and (2) site now can leverage the cloud power to scale out content delivery to the browser directly from the cloud storage.

    Currently the plugin works only with Amazon S3-compatible storage, other storage providers may be added in the near future. WP2Cloud plugin uses of ClouSE storage engine for MySQL to store media content in the cloud storage.

    WP2Cloud plugin source code and documentation are freely available for download.
    As the best deployment practice we recommend moving to the cloud the whole website, both its content (that is stored in WordPress database) and media files. This will allow you to take full advantage of cloud storage. Consider using Yapixx as a starting point and a reference of such configuration.

back to the top