From Eigenpedia

This page documents some experimental work being undertaken to bridge Farrago's local data wrapper SPI to allow MySQL storage engines to be plugged in.

NOTE jvs 11-Oct-2009: the experimental code mentioned here has been deleted from the head revision due to lack of interest, but it can be resurrected by sync'ing to eigenchange 13089.

The first cut is a proof of concept to directly bridge to a specific storage engine (ScaleDB). Followups would be to make this more robust and then introduce a generic storage engine interface (independent of both ScaleDB and MySQL).

Contents 1 Build 2 Configure 3 Run 4 Restore 5 Implementation

Build

For now, use Linux, not Windows.

After syncing the Eigenbase codebase as described in FarragoBuild, but before running initBuild.sh, perform the following steps to add in the ScaleDB libraries:

1. cd dev/fennel/seb
2. svn co http://scaledb-interfaces.googlecode.com/svn/trunk interface
3. obtain a copy of libscaledb.so and place it in the new seb/interface directory (for this binary, you'll need to talk to the ScaleDB folks)

Now run the Farrago initBuild.sh; it will automatically detect the presence of the ScaleDB interfaces and attempt to build the bridge code.

Configure

In dev/farrago/catalog, create a file scaledb.cnf, based on dev/fennel/seb/interface/docs/scaledb.cnf; edit all paths to be absolute paths rooted at some place we'll call data, e.g.

data_directory = /home/test/data/scaledb/

Then, use a script like this one to create the corresponding empty data directory structure:

cd /home/test
rm -rf data
mkdir data
mkdir data/scaledb
mkdir data/scaledb/sdb_system
mkdir data/scaledb/test
mkdir data/scaledb_logs

Run

In dev/farrago, create a script like the following, call it sqllineSeb, and chmod it to be executable:

#!/bin/bash
# $Id: //open/dev/farrago/sqllineEngine#3 $
# Run the sqlline command-line SQL interpreter 
# with an embedded Farrago engine

source ./defineFarragoRuntime.sh

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/dev/fennel/seb:/path/to/dev/fennel/seb/interface

if java ${SERVER_JAVA_ARGS} \
    -Dnet.sf.farrago.defaultSessionFactoryLibraryName=class:net.sf.farrago.defimpl.FarragoSebSessionFactory \
    ${SQLLINE_JAVA_ARGS} \
    -u jdbc:farrago: -d net.sf.farrago.jdbc.engine.FarragoJdbcEngineDriver \
    -n sa $*;
then
    echo
else
    tset
    echo "Terminal reset because sqlline crashed"
fi

Then use this instead of the normal sqllineEngine script so that the storage engine bridge will be used for tables instead of the default FTRS row-store.

You should be able to create a table, insert some rows, and select them back:

jvs@bagheera:~/open/dev/farrago$ ./sqllineSeb 
Connecting to jdbc:farrago:
Connected to: Farrago (version 0.1.0)
Driver: FarragoJdbcDriver (version 0.1)
Autocommit status: true
Transaction isolation: TRANSACTION_REPEATABLE_READ
sqlline version 1.0.7-eb by Marc Prud'hommeaux
0: jdbc:farrago:> create schema x;
No rows affected (0.608 seconds)
0: jdbc:farrago:> create table x.t(i int not null primary key, c char(10));
No rows affected (0.493 seconds)
0: jdbc:farrago:> insert into x.t values (1, 'hi');
1 row affected (0.858 seconds)
0: jdbc:farrago:> insert into x.t values (2, 'bye');
1 row affected (0.142 seconds)
0: jdbc:farrago:> select * from x.t;
+----+-------------+
| I  |      C      |
+----+-------------+
| 1  | hi          |
| 2  | bye         |
+----+-------------+
2 rows selected (0.1 seconds)

Some things won't work; for example, attempting to look up a row using an index will cause a crash since the corresponding access method hasn't been developed yet:

0: jdbc:farrago:> select * from x.t where i=1;
java: /home/jvs/open/dev/fennel/segment/RandomAllocationSegmentBase.cpp:380: virtual fennel::BlockId fennel::RandomAllocationSegmentBase::translatePageId(fennel::PageId): Assertion `const_cast<RandomAllocationSegmentBase *>(this)->isPageIdValid(pageId)' failed.
*** CAUGHT SIGNAL 6; BACKTRACE:
...

This is one way to verify that you're actually using the storage engine bridge :)

Restore

To restore to a clean database, e.g. after a crash, first run ant restoreCatalog from dev/farrago, then run the script from the #Configure instructions to delete and recreate the ScaleDB data directory.

Implementation

You can browse the C++ code and the Java code. It's very much quick-and-dirty (e.g. when a table is created, its ID is combined with the ID of the primary key index and stored as a fake PageID in the catalog); a lot of refactoring work has to take place to support the bridge for real.

• Currently, the only datatypes tested are the integer types of various width, and fixed-width CHAR. VARCHAR is not working correctly yet, nor are null values.
• A table has to have a primary key defined, but unique constraints and secondary indexes are not yet supported.
• Only sequential scans are supported.
• Savepoints are not yet supported. Concurrency is unknown until the correct binding between threads and user ID's has been discussed.
• ScaleDB API return values are not yet checked, so errors are silently ignored.