Using the SQL Relay drop-in replacement library for MySQL/MariaDB

What is a drop-in replacement library?

The SQL Relay Drop-In Replacement Library for MySQL/MariaDB is a shared-object library that can be LD_PRELOAD'ed to take the place of the native MySQL/MariaDB client library.

It allows many applications that are written to use MySQL/MariaDB directly to use SQL Relay without modification. This allows an app to immediately take advantage of SQL Relay features such as connection pooling, load balancing, query routing and throttling. It can also be used to aim an app that was written to use MySQL/MariaDB at a different database.

Using the drop-in replacement library with command-line programs

To use the SQL Relay drop-in replacement library for MySQL/MariaDB clients, you first need to determine what version of MySQL/MariaDB your existing application was compiled against. There are 4 different drop-in replacement libraries. One each for MySQL versions 3.xx, 4.0, 4.1, 5.0 and 5.1 or higher. MySQL's native data structures changed a bit between the different versions. If you use the wrong library your program will most likely crash.

You can find out which library is installed on your system by running the following command.

/usr/bin/mysql --version

It should return something like the following:

mysql  Ver 14.14 Distrib 5.1.60, for redhat-linux-gnu (i386) using readline 5.1

The number after Distrib is the version.

Below is a chart mapping version numbers and which library to use:

• 3.X - libmysql3sqlrelay.so
• 4.0 - libmysql40sqlrelay.so
• 4.1 - libmysql41sqlrelay.so
• 5.0 - libmysql50sqlrelay.so
• 5.1 or higher - libmysql51sqlrelay.so
• any version of MariaDB - libmysql51sqlrelay.so

Once you've figured out which version of MySQL/MariaDB is installed on your system, you can load the appropriate drop in library and run your program.

The parameters that would ordinarily indicate which host, port, socket, username and password to use to connect to MySQL/MariaDB will be used as parameters to connect to SQL Relay. The parameter that would ordinarily indicate which database to connect to will be ignored. Instances of SQL Relay are configured to connect to a single database, and that database will be used by the client program.

In the following example, we're running the "mysql" program against an instance of SQL Relay running on the localhost, port 9000 against an Oracle database. This instance of SQL Relay is configured with a username/password of oracleuser/oraclepass.

For sh-based shells:

LD_PRELOAD=/usr/local/firstworks/lib/libmysql51sqlrelay.so.4
export LD_PRELOAD
mysql -h localhost -P 9000 --user=oracleuser --password=oraclepass

For csh-based shells:

setenv LD_PRELOAD /usr/local/firstworks/lib/libmysql51sqlrelay.so.4
mysql -h localhost -P 9000 --user=oracleuser --password=oraclepass

The LD_PRELOAD environment variable instructs the dynamic loader to load libmysql51sqlrelay.so before loading any other libraries for any programs. The mysql client program will still load the native MySQL/MariaDB client library, but since it loaded the SQL Relay drop-in replacement library first, function calls that would normally be fulfilled by the native MySQL/MariaDB client library are fulfilled by the SQL Relay drop-in replacement library instead.

If your application was compiled against a different version of MySQL/MariaDB, you will need to replace libmysql51sqlrelay.so with the appropriate library.

Below is a sample session using the mysql command line client against an Oracle database through SQL Relay.

(NOTE: As of version 5.5, the mysql command line client appears to have been statically linked, at least in some distributions. As a result, the LD_PRELOAD trick doesn't work with it, but should still work with any program that is dynamically linked against libmysqlclient.)

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 0 to server version: 3.23.58

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> create table exampletable (col1 varchar2(60), col2 number(5,2));
Empty set (0.21 sec)


mysql> insert into exampletable values ('hello',123.45);
Empty set (0.00 sec)


mysql> select * from exampletable;
+-------+--------+
| COL1  | COL2   |
+-------+--------+
| hello | 123.45 |
+-------+--------+
1 row in set (0.09 sec)


mysql> drop table exampletable;
Empty set (0.21 sec)


mysql> quit;

Using the drop-in replacement library with daemons

Using the SQL Relay drop-in replacement library for MySQL/MariaDB with daemons is simlar to using it on the command line. You just need to add the LD_PRELOAD command to the startup script for the daemon before the command that starts the daemon itself.

If your system uses systemd instead of init scripts, then you can modify the systemd configuration to make the LD_PRELOAD setting part of the environment. For example, the Apache webserver is typically started by the systemd config file /lib/systemd/system/httpd.service. To configure Apache, create a new file /etc/systemd/system/httpd.service with the following contents:

Using the drop-in replacement library with inetd/xinetd helper programs

Inetd and xinetd are daemons that listen on ports and run helper programs to service requests on those ports. The helper programs die off after the request is serviced.

The easist way to get an inetd helper program to use the SQL Relay drop-in replacement library for MySQL/MariaDB is to add the LD_PRELOAD command to the startup script for inetd/xinetd. Any command that inetd/xinetd runs will also preload the library.

However, if some of the helper programs need to actually run against MySQL/MariaDB and not against SQL Relay, then you will have to do something different. The easiest thing to do is create a script for each helper program that needs to run against SQL Relay that runs the LD_PRELOAD command and then runs the actual helper program, passing it all the necessary command line arguments.

For example, lets say you have a pop3 server called pop3d that uses MySQL/MariaDB for user identification and you wanted to use SQL Relay instead of MySQL/MariaDB. The inetd.conf entry might look like this:

pop3 stream tcp nowait root /usr/local/bin/pop3d

An /etc/xinetd.d entry might look like this:

service pop3
{
	socket_type	= stream
	wait		= no
	user		= root
	server		= /usr/local/bin/pop3d
}

You could write the a script called /usr/local/bin/pop3d-sqlrelay as follows:

#!/bin/sh
LD_PRELOAD=/usr/local/firstworks/lib/libmysql51sqlrelay.so.4
export LD_PRELOAD
/usr/local/bin/pop3d $@

And modify the entries to call the script instead of pop3d as follows:

pop3 stream tcp nowait root /usr/local/bin/pop3d-sqlrelay

Or for xinetd:

service pop3
{
	socket_type	= stream
	wait		= no
	user		= root
	server		= /usr/local/bin/pop3d-sqlrelay
}

Using the drop-in replacement library with PHP

Ideally, if you wanted to use the drop-in replacement library for MySQL/MariaDB with PHP, you'd just add the LD_PRELOAD command to the startup script for the http daemon, but PHP is not typically linked against the MySQL/MariaDB client library, rather the PHP source code includes it's own MySQL/MariaDB client API which is compiled directly into the PHP module itself.

So, to use the drop-in replacement library with PHP, you must rebuild PHP and tell it to link against the MySQL/MariaDB client library.

Using the drop-in replacement library with modules

You may want to use the SQL Relay drop-in replacement library for MySQL/MariaDB clients with a program that isn't compiled against the native MySQL/MariaDB client library but rather loads it as a module such as a program that uses ODBC or Perl DBI, or an Apache/PHP application.

Using the SQL Relay drop-in replacement library with programs that load the native MySQL/MariaDB client library as a module is simlar to using it on the command line. You just need to make sure that the LD_PRELOAD command is run before the program starts.

If the program is a command line program, then run the LD_PRELOAD command before running your program. Even though the program ultimately loads the native MySQL/MariaDB client library, all of its functions will be overriden by the SQL Relay drop-in replacement library.

If the program is a daemon then add the LD_PRELOAD command to the startup script or systemd configuration for the daemon.

If the program runs in the address space of a daemon, such as a PHP application running under Apache's mod_php, then add the LD_PRELOAD command to the startup script or systemd configuration for the daemon. The caveat here is that all applications running in the address space of the daemon will use the drop-in replacement library instead of the native MySQL/MariaDB library. It is not possible, for example for a web server to run one PHP application directly against MySQL/MariaDB and another PHP application against SQL Relay using the drop-in replacement library; if the drop-in replacement library is loaded, both applications will end up using it.

Function support

The SQL Relay drop-in replacement library for MySQL/MariaDB implements most of the native MySQL/MariaDB client library's functions, but there are a few functions that aren't implemented because SQL Relay doesn't d way to support them. These functions return safe values or a failure condition.

Tweaks

When fetching a date into a MYSQL_TIME struct, the drop-in library attempts to interpret whatever it got back from the database as a date. It can parse the native formats of all the databases that SQL Relay currently supports, but it cannot tell by examination whether a date like 02-05-2011 is in MM-DD-YYYY format or DD-MM-YYYY format. By default, it assumes MMDD, but if you want it to assume DDMM instead, set the environment variable SQLR_MYSQL_DATE_DDMM="yes". Another environment variable allows you to specify different behaviors for dates that start with years, such as 2011-02-05. If SQLR_MYSQL_DATE_YYYYDDMM="yes" then that date would be interpreted as being in the YYYY-DD-MM format. If it is set to "no" then that date would be interpreted as being in the YYYY-MM-DD format. If it is not set, then it defaults to whatever SQLR_MYSQL_DATE_DDMM is set to. If neither are set then dates are universally interpreted to be in MM/DD format. There are very specific cases where these two environment variables need to be set differently from one another. You'll know if you need to.

When using mysql_stmt_bind_result, it's possible to bind fewer buffers than there are columns in the result set. For example you might run "select col1, col2, col3 from mytable" but only bind a single value, expecting col1 to be fetched and the others ignored. The native MySQL/MariaDB library doesn't support this but some apps do it anyway and it's just fortune that they don't crash. Some apps try a little harder and "NULL-terminate" the bind array with a zeroed-out bind buffer. This doesn't appear to have any effect with the native MySQL/MariaDB library but with the drop-in replacement library if you set the environment variable SQLR_MYSQL_NULL_TERMINATED_RESULT_BINDS="yes" then mysql_stmt_fetch() and mysql_fetch() will stop populating bind buffers when they find one containing NULL's for the buffer, length and is_null pointers.

Mapping Error Codes

By default, mysql_stmt_errno and mysql_errno return the native error code from the database. For example, if SQL Relay is pointed at Oracle and a query is missing a FROM clause, then the Oracle error code 923 will be returned - the error code corresponding to the Oracle error "FROM keyword not found where expected" rather than the MySQL/MariaDB error code 1064 corresponding to the MySQL/MariaDB error "ER_PARSE_ERROR".

Since various databases have thousands of error codes, it's not feasible to map every one to the corresponding MySQL/MariaDB error code. It is rare for an app to react to specific error codes, but if your app does, and requires specific MySQL/MariaDB error codes to be returned, then there it is possible to map native error codes to MySQL/MariaDB error codes.

To do this, you must set the environment variable SQLR_MYSQL_ERROR_MAP to the location of an error map file. For example:

export SQLR_MYSQL_ERROR_MAP=/usr/local/firstworks/etc/oracle-mysql-errors.map

This must be set on the client side - on the machine and in the shell where the drop-in replacement library is being run.

Example error map files for Oracle and Microsoft SQL Server are provided at /usr/local/firstworks/share/examples/sample-oracle-mysql-errors.map and /usr/local/firstworks/share/examples/sample-mssqlserver-mysql-errors.map.

The format is described at the top of the file and examples follow:

# sample oracle to mysql error mappings...
# format: <oracle error number>:<mysql error number>

# FROM keyword not found where expected... -> ER_PARSER_ERROR
# (oracle actually has lots of syntax errors and they all need to be mapped to
# ER_PARSE_ERROR)
923:1064

# quoted string not properly terminated -> ER_PARSE_ERROR
1756:1064