Release Notes

Release notes for ClouSE v1.0

Special thanks to 2K+ Beta users and early adopters who used ClouSE and provided feedback, so that we could make it better. We wouldn’t be able to release such a great product without you!

Changes in ClouSE v1.0.2.1, released 07/01/2014

Note that automatic data upgrade from version 1.0b.1.x is not supported. To migrate data to version (or later), the data needs to be exported using the mysqldump utility, and then re-imported to a new cloud location. The update-clouse script can detect this condition and re-import the data automatically.

Commits 7f43dd, d48457

Implement licensing. Now a license can be set via clouse_license_* session variables. Licensing information can be retrieved via clouse_license_* status variables. To see the current licensing information the SHOW STATUS LIKE 'clouse_license_%' SQL statement can be used. ClouSE license can be ordered at Should you have any questions about licensing don’t hesitate to contact us.

Commits 44b4b7, 378cf2, c2719a, 5c3490, 7b29b3, b728ab

Add support for MySQL 5.5.35 – 5.5.38 and MySQL 5.6.16 – 5.6.19. Drop support for MySQL prior to 5.5.35 and MySQL prior to 5.6.16. If you need a ClouSE build for MySQL prior to 5.5.35 or 5.6.16 please contact us.

Changes in ClouSE v1.0rc.2.0, released 01/15/2014

Note that automatic data upgrade from version 1.0b.1.x is not supported. To migrate data to version 1.0rc.2.0 (or later), the data needs to be exported using the mysqldump utility, and then re-imported to a new cloud location. The update-clouse script can detect this condition and re-import the data automatically.

Commit 85d4f4

Add support for automatic data upgrades starting from version 1.0.2.x (this version). Now newer versions of ClouSE are required to support old data formats. Automatic data upgrades from data version 1.0.1.x (v1.0 Beta) are not supported.

Commit 4c5a95

Make internal representation of VARCHAR use 1 or 2 bytes to store string size. Previously a fixed size of 4 bytes was used.

Commit 07b3d4

Reduce the length of Weblob URLs for usability. The random salt is reduced from 256-bit to 128-bit.

Commit ff5b2b

Add support for the AUTO_INCREMENT table option in the CREATE TABLE and ALTER TABLE statements.

Commits 76c785, fe3882

Add configuration options to control on-disk cache for relational data. The clouse_local_data_path option can be used to specify the on-disk cache path. If the path is not specified, the cache files are created in the MySQL data directory with the clse_ prefix. The clouse_local_data_size option can be used to specify the maximum cache size. If the maximum on-disk cache size is not specified, the size is auto-calculated based on the available free disk space. To disable on-disk cache, the clouse_local_data_size option can be set to 0.

Commits 21e8fd, 34d810, 2a3cc6, 71bfe2

Implement persistent on-disk cache for relational data. The on-disk cache data persists across server restarts and machine reboots, it is crash-tolerant and self-healing. If the amount of data exceeds the maximum cache size, ClouSE tries to keep the most frequently used pages in the cache. With persistent on-disk cache in place, the amount of data reads from the cloud storage is drastically reduced. This removes the last Beta limitation: persistent on-disk cache of recently accessed relational data is not implemented.

Commits a6d57d, 48a51b, 921800

Implement automatic recovery from cloud. Previously ClouSE would go offline and wait for an explicit user action to start the recovery; automatic recovery was unsafe and could lead to data corruption in some cases. Now with automatic full history retention, automatic recovery from cloud can property recover data in all cases. The recoverlogfromcloud ClouSE admin command is still accepted (for backward compatibility) but doesn’t do anything.

Commits 942374, af1c8c

Implement point-in-time recovery. The new restoreto YYYY-MM-DD hh:mm:ss+tztz ClouSE admin command (usage example: SET GLOBAL clouse_admin_command='restoreto 2014-01-15 17:03:34-0800';) can be used to restore to any point in time within the retention period with second granularity. The timezone offset must be explicitly specified to avoid confusion between the user’s and server’s timezones. The restoreto command is not destructive and its effects can be undone by recovering to the point in time just before the comand was issued. During point-in-time recovery operation the engine can be in the Restoring state, which is reflected in the SHOW ENGINE CLOUSE STATUS; command.

Commit a4096e

Add clouse_retention_period configuration option. The option can be used to configure history retention period. The retention period can be configured from 0 to 744 hours (31 days). The default value is 24 hours.

Commits bd6983, acb33e, 932cb1, afe171, fa0ad7, 3640f4, 306417, 3858ab, c09af2

Implement automatic full history retention. Now the database can be restored to any point in time within the retention period. Full history retention is achieved with virtually no overhead during normal processing; the only effect is that log truncation and Weblob removal operations are deferred for the retention period.

Changes in ClouSE v1.0b.1.10, released 11/25/2013

Commit a17914

Implement statistics collection for tables / indexes and provide the statistics to MySQL’s query optimizer. Previously ClouSE provided hardcoded statistics that did not represent the actual data and could lead to suboptimal query plans. This removes the following Beta limitation: the statistics for ClouSE tables are not implemented..

Changes in ClouSE v1.0b.1.9, released 10/05/2013

Commit 931d57

Add support for MySQL 5.5.32-34. Add support for Cloud Linux flavor of MySQL: 5.5.31-cll – 5.5.33-cll. Drop support for MySQL 5.5.19 – 5.5.23. If you need a ClouSE build for MySQL prior 5.5.24 please contact us.

Commit 1766d3

Add clouse_max_weblob_download_size configuration option. The option can be used to specify the maximum size of Weblob content that can be retrieved via SELECT statement. The maximum size can be configured from 1 KB to 1 GB, and the default size is 16 MB. Previously the maximum size was 16 MB and could not be configured by the user.

Commit a15208

Accept broken certificate chains, as long as the certificate chain is correctly rooted in the trusted CA certificate store. The standard OpenSSL logic requires that the certificate chain reported by the server either ends with a self-signed certificate, or ends with a certificate an issuer for which could be found in the trusted CA certificate store. At the time of this release reported a broken certificate chain and the standard OpenSSL certificate validation failed. This is now fixed.

Changes in ClouSE v1.0b.1.8, released 06/02/2013

Commit 305c71

Previously the errors that were encountered on ClouSE startup would be logged into the MySQL’s error log only. Now in addtion to being logged into the error log, ClouSE startup errors are shown to the user who tries to access ClouSE tables.

Commit 86555d

Previously cache control directives for Weblobs were not set, which limited the effectiveness of proxy / CDN caches. Now Weblobs have the max-age cache control directive set to allow the content to be cached for one year. The fix also affects WebStor, which now provides a way to set cache control directives.

Commits da5680, cb1c93

Add support for MySQL 5.5.31. Add support for Cloud Linux flavor of MySQL: 5.5.30-cll.

Changes in ClouSE v1.0b.1.7, released 02/27/2013

Commit cf1491

Add support for Percona Server 5.5.29-rel30.0. Percona Server is supported only on Linux. The handlerton library for Percona Server is

Commit 172b84

Add support for MySQL 5.5.30 and 5.6.10. Now ClouSE supports both MySQL 5.5.x and 5.6.x release series.

Commit 0f9e00

Add support for MySQL partitioning. Partition’s metadata is saved to the cloud storage and recovered on auto-discovery. Subpartitions are currently not supported. If you need ClouSE to work with subpartitions please contact us.

Commit 6472a9

Add support for ClouSE on Microsoft Windows. ClouSE builds are now available both for Linux and Windows. The cloud data format is the same across all platforms (Windows, Linux, 32-bit, 64-bit).

Commit 077e5d

Implement auto-discovery logic for missing tables when recovering from cloud storage. Auto-discovery has always worked for existing tables: the table’s .frm file gets transparently recovered from the cloud storage on first access. This change makes it so that the table’s .frm file is transparently removed if the table is missing in the cloud storage.

Commit 1025d3

Previously under stress ClouSE could hang while accessing cloud storage. This is now fixed.

Changes in ClouSE v1.0b.1.6, released 12/24/2012

Commit a6f921

Add support for MySQL 5.5.29. Drop support for MySQL 5.5.17 – 5.5.18. If you need a ClouSE build for MySQL prior 5.5.19 please contact us.

Commit 03fa7b

Implement support for DECIMAL fields in keys. This removes the following Beta limitation: DECIMAL fields cannot be used in keys.

Commit fcc55c

Previously under stress ClouSE could run out of memory due to a memory leak. This is now fixed. The fix also affects WebStor.

Commit f764e5

Implement support for tables without a primary key. This removes the following Beta limitation: The table must have a primary key.

Commit ab62c2

Previously NULL values were not properly stored if used in FLOAT fields. This is now fixed.

Changes in ClouSE v1.0b.1.5, released 11/21/2012

Commit 10c9e4

Allow MySQL query cache to cache query results in auto-commit mode.

Commit 3da996

Previously the query optimizer could choose a full table scan even when it was much more efficient to use an index. SELECT queries that have WHERE clause with IN (expr [, expr] ...) conditions were most likely to incur a performance hit. This is now fixed.

Commit 4e4caa

Implement support for BLOB fields larger than 128 KB. Now BLOB fields can have any size that is supported by MySQL. Small BLOB fields continue to be stored in the record for efficiency (so there is virtually no penalty for using a BLOB instead of a VARCHAR(N)). The on-disk data format is backward compatible, and the previous versions of ClouSE would gracefully fail if they encounter BLOBs created by ClouSE v1.0b.1.5. This removes the following Beta limitation: BLOB fields are stored in the record thus are limited by the max record size (128 KB).

Commit f68411

Make Weblob operations more resilient to intermittent cloud storage connection failures.

Commits 0a4889, 1723b4, 242b5c

WebStor: make WebStor cloud storage performance benchmark (wsperf) print correct statistics.

Commits 677878, 40e455

WebStor: clean up codebase, error messages and documentation to consistently use general cloud storage terminology instead of Amazon S3. The WebStor API now accepts 3 storage types: WST_S3, WST_GCS, WST_WALRUS. The s3test utility is renamed into wscmd to better reflect its purpose.

Commit a0f640

Add support for Google Cloud Storage as a cloud storage provider. Now a Google Cloud Storage location can be specified in the clouse_cloud_data_url configuration option using the gs:// protocol scheme, e.g.

Changes in ClouSE v1.0b.1.4, released 10/19/2012

Commit 1f20ec

Add support for MySQL 5.5.27 – 5.5.28. Drop support for MySQL 5.5.14 – 5.5.16. If you need a ClouSE build for MySQL prior 5.5.17 please contact us.

Commit 18bd3c

Implement NULL support for all data types. The on-disk data format is backward compatible, and the previous versions of ClouSE would gracefully fail if they encounter tables with NULLable fields created by ClouSE v1.0b.1.4. This removes the following Beta limitation: Integer, date / time, decimal, float, bit, enum, and set fields must be declared NOT NULL.

Commit 622f05

Implement TRUNCATE statement.

Changes in ClouSE v1.0b.1.3, released 8/22/2012

Commit e32f5e

Previously some SELECT statements (e.g. using index_merge w/intersect query execution plans) may mistakenly miss records. This is now fixed.

Changes in ClouSE v1.0b.1.2, released 6/28/2012

Commit 1f424e

Add support for MySQL 5.5.22 – 5.5.25. Drop support for MySQL 5.5.8 – 5.5.13. If you need a ClouSE build for MySQL prior 5.5.14 please contact us.

Commit 1c3868

Implement support for Weblob content type. Previously, all Weblobs were uploaded with the application/octet-stream content type, which could result into Web Browsers’ showing the Weblob content incorrectly (e.g. a Web Browser could try to save an image file to disk instead of showing the image). Now the content type is set according the the Weblob name (using a name-to-content-type mapping derived from this document).

Commits 055291, 2413dc, c6fc3b, dee2c9, 683d40, 268705, ea5483, a38ba2

Implement miscelaneous performance optimizations.

Changes in ClouSE v1.0b.1.1, released 4/19/2012

Commit aada72

Implement high-performance cloud storage access layer. The functionality is also released as WebStor – the high performance open source C++ library for Amazon S3.

Changes in ClouSE v1.0b.1.0, released 3/06/2012

Commit 909582

Add support for MySQL 5.5.20 and 5.5.21.

Commit 5490be

Add support for specifying weblob names instead of just extensions. Ability to specify names is important for SEO, so now instead of ‘extension:jpg’ the whole name can be specified as ‘name:creative-thinking.jpg’. This is a breaking change.

Commit 3d9cd1

Change versioning structure: split feature version into a major.minor pair, so instead of 1b.1.0 the new version is 1.0b.1.0.

Commit 143d8a

Increase data page size from 64 KB to 256 KB to improve network bandwidth utilization on high-latency connections.

Commit d2a432

Implement AUTO_INCREMENT semantics similar to InnoDB’s “interleaved” lock mode. Previously, AUTO_INCREMENT was implemented as a global monotonically increasing counter; the existing values in the table were not taken into account. This could lead to auto-incremented values overriding existing values in the table. Now the auto-incremented values are picked to be higher than any existing value in the table.

Commit 09c143

Now weblobs must have the LONGBLOB type. Allowing BLOB type could lead to confusion when weblob streaming was used: the content was correctly uploaded in small chunks, but the result got truncated by the SELECT statement on retrieval.

Commit a6ae82

Previously under stress the buffer manager could excessively write pages to the cloud storage. This is now fixed.

Commit 1fa5ab

Previously the transaction commit grouping logic was not effective in some cases. This is now fixed.

Commit 2fcd83

Implement checkpoint on shutdown. The checkpoint is not required for proper functioning (crash at any point is a fully supported scenario), but may help with restart performance.

Commit 6d6841

Implement more aggressive checkpoint flushing. Previously checkpoints were completely lazy and didn’t flush the log to the durable storage.

Commit 4a4f97

Implement “sorted split” logic for B-trees. Now when a record is inserted into the end of a B-tree the page is split 90/10% instead of 50/50%. This is a standard B-tree optimization that was cut from the initial Beta release.

Commit 520c5a

Previously under stress ClouSE could crash due to a double-delete problem in the lock manager. This is now fixed.

Commit 0ad2ec

Previously lock escalation logic might not properly kick in, leading to excessive memory usage and potentially out-of-memory errors. This is now fixed.

Changes in ClouSE v1b.0.0, released 1/12/2012

Initial relase of ClouSE v1.0 Beta.