RSpace 1.69.33 Update Notes

Updated 6 months ago by RSpace Support

This page is only relevant to on-premises customers who maintain RSpace installations themselves.

This article deals with an older version of RSpace. Before installing or updating any on-premise RSpace server, always refer to the version-specific .md documentation files included with the download bundle for that version. Contact ResearchSpace for the download URL.

In RSpace 1.69.33 we are implementing a series of database refactorings to move image files out of the database and into the RSpace file-store.

This will bring several advantages:

The database will be much smaller, making backup and recovery much faster.
More of the database will be able to be held in memory, resulting in better performance.

The refactoring will occur in several steps, starting with 1.69.33. This initial step will copy files from the database to the RSpace file-store, but not delete them from the database.

Before updating to 1.69.33, please read the following procedure, and contact Research Space if you have questions or concerns.

Planning steps before updating to 1.69.33

The code snippets below may have to be adjusted for your own database name, credentials and paths.

1. Run some database queries to assess how much data will be migrated

mysql -uusername -p rspace

mysql> select count(*) from EcatImage;

mysql> select count(*) from EcatDocumentFile;

If you have less than 10 000 (ten thousand) each of these items, the upgrade will be unlikely to produce any resource issues, and you can disregard step 2.

If you do have more, please do the following:

2. Making sure you have sufficient resources for the migration

Check if binary logging is enabled:
mysql> show variables like '%log_bin%';
If it’s on, turn it off by commenting out the following line in /etc/mysql/mysql.conf.d/mysqld.cnf:
#log_bin = /var/log/mysql/mysql-bin.log
Disabling binary logging will prevent the generation of enormous log files and speed up the migration process.
Ensure enough space for MySQL tmpdir. The default location is /tmp. If the temp folder is on the same drive as the main data folder no change should be needed.
mysql> show variables like 'tmpdir';
If you've edited any MySQL variables in the configuration file, restart MySQL before starting the migration.
Look in your MySQL data directory to see the size of the ImageBlob.frm table, eg. sudo ls -lrt /var/lib/mysql/rspace
Please ensure there is at least 5x this capacity available on the system before starting.
Increase Java heap size if need be in /etc/default/tomcat8. In JAVA_OPTS variable, edit -Xmx to a value at least 2048m.
If -Xmx is not set, there is no need to set it - Java memory will scale to the available system memory

Performing the update to 1.69.33

We recommend setting up a maintenance window of 30 minutes; in most cases the migration will complete sooner.

If you have more than 10000 images in the database, check that MySQL variables are set correctly (see preparatory steps above).

Note the version of RSpace you are currently using (in case you have to revert).
Stop the RSpace server. Copy the existing .war application file to backup:
sudo cp /var/lib/tomcat8/webapps/ROOT.war ~/rsp-pre169-33.war
Either make a cold image snapshot, or a logical mysql database dump:
mysqldump -uusername -p rspace --single-transaction --ignore-table rspace.ImageBlob_AUD --ignore-table rspace.FieldAutosaveLog > rspac-pre169-33.sql
Update RSpace as usual. The migration runs asynchronously, after RSpace starts up. Users can access RSpace while the migration is running, although performance may be reduced.
The migration is complete when you see a log line similar to INFO /12/14 18:46:liquibase: Successfully released change log lock appearing for the second time.

Depending on the speed of your disk, expect to be able to migrate about 30000-40000 files per hour.

If RSpace crashes or runs out of memory, restart RSpace - it will continue the migration from where it left off. If it repeatedly runs out of memory, increase the available heap size (step 2.6).

Post-migration steps

Restore MySQL binary logging, if was previously enabled (in step 2.2).

Asserting the migration worked

All these queries should return an empty result set.

select count(*) from failedImgWorkingConversions;

select count(*) from assertImgWorkingSize where blobLength != fpSize;

select count(*) from failedImgThumbnailConversions;

select count(*) from assertImgThumbnailSize where blobLength != fpSize;

select count(*) from assertThumbnailSize where blobLength != fpSize;

select count(*) from assertDocThumbnailSize where blobLength != fpSize;