From Eigenpedia

Contents 1 FAQ List 1.1 Startup Error for libgcj 1.2 Startup Error for xlib xcb 1.3 Startup Error For libaio 1.4 Startup Error For mmap 1.5 Startup Error For Windows 1.6 Client Connectivity Problems 1.7 Loading Large Data Files 1.8 ETL Tutorial 1.9 Case-Sensitive Foreign Schema Name 1.10 Default Port 1.11 JDBC Driver Class 1.12 Slow INSERT performance 1.13 Missing Columns 1.14 Running As A Service Or Daemon 1.15 J2EE Embedding 1.16 Catalog File Locking 1.17 Debug Builds 1.18 Beyond Java 1.19 Compression Statistics 1.20 OpenSUSE Performance 1.21 Identifier Quoting

FAQ List

This page is the FAQ list for LucidDB users. For questions about the project, see the project FAQ on the website.

Startup Error for libgcj

Question

I got an error about initializeClass on startup. How do I fix it?

Exception in thread "main" java.lang.NoClassDefFoundError: 
net.sf.farrago.db.FarragoDatabase
    at java.lang.Class.initializeClass(libgcj.so.90)
    at 
net.sf.farrago.db.FarragoDbSingleton.pinReference(FarragoDbSingleton.java:100)
    at 
net.sf.farrago.server.FarragoAbstractServer.start(FarragoAbstractServer.java:232)
    at com.lucidera.farrago.LucidDbServer.main(LucidDbServer.java:61)

Answer:

The GCJ virtual machine (packaged by default with many Linux distributions) is not sufficient for use with LucidDB. Instead, please install a Sun Java JVM, set your JAVA_HOME to the Sun JRE, then do a fresh unzip/install.

Startup Error for xlib xcb

Question:

I got an error similar to one of the ones below on startup. How do I fix it?

java: xcb_xlib.c:50: xcb_xlib_unlock: Assertion `c->xlib.lock' failed.

Locking assertion failure. Backtrace:
#0 /usr/lib/libxcb-xlib.so.0 [0xb7361767]
#1 /usr/lib/libxcb-xlib.so.0(xcb_xlib_unlock+0x31) [0xb73618b1]
#2 /usr/lib/libX11.so.6(_XReply+0xfd) [0x99e591bd]
#3 /usr/lib/jvm/java-1.5.0-sun-1.5.0.16/jre/lib/i386/xawt/libmawt.so [0x9b539dce]
#4 /usr/lib/jvm/java-1.5.0-sun-1.5.0.16/jre/lib/i386/xawt/libmawt.so [0x9b523d77]
#5 /usr/lib/jvm/java-1.5.0-sun-1.5.0.16/jre/lib/i386/xawt/libmawt.so [0x9b523ef3]
#6 /usr/lib/jvm/java-1.5.0-sun-1.5.0.16/jre/lib/i386/xawt/libmawt.so(Java_sun_awt_X11GraphicsEnvironment_initDisplay+0x26) [0x9b524136] 

Answer:

The easiest way is to edit the bin/lucidDbServer startup script and add -Dorg.eigenbase.util.AWT_WORKAROUND=off to the Java command line. (This will be done as part of the out-of-the-box packaging in the next release since this seems to be a common problem.)

See this mailing list thread for more discussion.

Startup Error For libaio

Question:

I got the error below on startup. How do I fix it?

Loading database...
Exception in thread "main" org.eigenbase.util.EigenbaseException: Failed to load database
    at net.sf.farrago.resource.FarragoResource$_Def1.ex(FarragoResource.java:1789)
    at net.sf.farrago.db.FarragoDatabase.<init>(FarragoDatabase.java:269)
    at net.sf.farrago.db.FarragoDbSingleton.pinReference(FarragoDbSingleton.java:89)
    at net.sf.farrago.server.FarragoAbstractServer.start(FarragoAbstractServer.java:183)
    at com.lucidera.farrago.LucidDbServer.main(LucidDbServer.java:61)
Caused by: org.eigenbase.util.EigenbaseException: FennelResource.en_US.libaioRequired()
    at net.sf.farrago.resource.FarragoResource$_Def0.ex(FarragoResource.java:1766)
    at net.sf.farrago.fennel.FennelDbHandle.handleNativeException(FennelDbHandle.java:315)
    at net.sf.farrago.fennel.FennelDbHandle.executeCmd(FennelDbHandle.java:242)
    at net.sf.farrago.fennel.FennelDbHandle.<init>(FennelDbHandle.java:90)
    at net.sf.farrago.db.FarragoDatabase.loadFennel(FarragoDatabase.java:509)
    at net.sf.farrago.db.FarragoDatabase.<init>(FarragoDatabase.java:189)
    ... 3 more

Answer:

On a Debian-based system such as Ubuntu:

apt-get install libaio1

(on older versions it may be libaio instead of libaio1)

On A RedHat-based system:

yum install libaio1

After that, you should be able to start the server. If you keep the default setting of aioLinux, you'll want to increase LucidDB's buffer pool size, and also preallocate data files, as mentioned in LucidDbPerformanceTuning#I.2FO_Scheduler.

Startup Error For mmap

Question:

I got the error below on startup, or when trying to increase the buffer pool size. How do I fix it?

Error: System call failed:  mmap failed: Cannot allocate memory (state=,code=0)

Answer:

Assuming you actually have sufficient physical memory, and are using a 64-bit build if trying to allocate a large buffer pool, check these:

1. Make sure ulimit -v returns the string "unlimited". Setting a quota on virtual memory size can cause the error above.
2. Make sure you aren't accidentally using a debug build. Debug builds use mprotect to guard against buffer overruns, and this eats up a lot of additional memory and OS page table space.

Startup Error For Windows

Question:

I got an error trying to load the native code DLL's on startup. How do I fix it?

Answer:

Make sure you have installed the Visual C++ 2008 Redistributable Package.

Client Connectivity Problems

Question:

I can't get sqllineClient, Squirrel, and other clients to talk to the LucidDB server. What's wrong?

Answer:

You may need to follow the instructions in ClientServerLocalhost to fix /etc/hosts.

Loading Large Data Files

Question:

I have a large data file in txt format. I need to bulk load it into Lucid tables. What's the best way to do this?

Answer:

Create a flat file foreign table corresponding to your flat file, and then use an INSERT-SELECT statement to load from the flat file table to your target table.

See LucidDbFlatFileWrapper for details.

LucidDbEtlTutorial#Define_Sources also has an example that shows you how to create a flatfile foreign data wrapper, import it, and then access tables within the imported foreign schema.

ETL Tutorial

Question:

In attempting to follow the ETL Tutorial I got an error because directory examples/etl/filedata/ does not exist. What should I do?

Answer

Make sure you followed the instructions to start the server like this:

bin/lucidDbServer

The tutorial relies on this for the relative paths to come out right. Otherwise, you need to edit the scripts so that they supply absolute paths.

Case-Sensitive Foreign Schema Name

Question:

I'm trying to import a foreign schema from a server like PostgreSQL with case-sensitive identifiers, and I'm not getting any tables back. How come?

Answer

Make sure you quote the foreign schema name in the IMPORT FOREIGN SCHEMA command:

import foreign schema "public"
from server postgres_abc
into ldb_schema 

Likewise for foreign tables.

Default Port

Question:

What is the default listening port number for LucidDB?

Answer:

By default, LucidDB listens for HTTP connections on port 8034. If you want to specify it explicitly from the client side, use a URL like

jdbc:luciddb:http://1.2.3.4:8034

You can change a server's port number by setting system parameter serverHttpPort; see LucidDbSystemParameters for more information.

JDBC Driver Class

Question:

I'm trying to connect to LucidDB from my own program, but I always get an exception like java.lang.ClassNotFoundException: org.luciddb.jdbc.LucidDbClientDriver. What should I do to fix it?

Answer:

Make sure you have luciddb/plugin/LucidDbClient.jar on your program's classpath.

Question:

I'm trying to use LucidDbLocalDriver to access the server from a client program running on the same (local) machine, but it doesn't work. Why not?

Answer:

The "local" driver is local only in the sense "in the same java process", not in the sense "on the same machine", so you'll need to use the HTTP driver. Use driver class org.luciddb.jdbc.LucidDbClientDriver unless the client and server are actually running in the same JVM.

Slow INSERT performance

Question:

I'm trying to INSERT lots of rows using the INSERT command, but the performance is terrible! What can I do to improve the performance?

Answer:

There are inherent issues with single row INSERTs in a column store database. Bulk INSERTs or UPDATEs are best accomplished (with orders of magnitude increases in performance) via a bulk loading method (see LucidDbEtlTutorial for an example).

Missing Columns

Question:

I created a wide table, and some of the columns seem to have gone AWOL. What gives?

Answer:

If you're using sqlline, see this mailing list thread.

Running As A Service Or Daemon

Question:

How do I run LucidDB as a service on Windows or a daemon on Unix?

Answer:

See LucidDbAsDaemonService.

J2EE Embedding

Question:

How do I run LucidDB inside of a J2EE app server?

Answer:

See LucidDbAsManagedBean.

Catalog File Locking

Question:

I got an exception about "Failed to lock catalog file" on server startup; what does this mean?

Answer:

It means you already have a server process running from the same installation, and only one can be running at a time. See this mailing list thread. For information on how to start LucidDB on a remote server, see this thread.

Debug Builds

Question:

I built LucidDB from source and now it's really slow. How come?

Answer:

By default, initBuild.sh produces an unoptimized debug build, which is an order of magnitude slower than release builds. Be sure to specify the --with-optimization --without-debug flags to match the release configuration, unless you're actually planning to hack on LucidDB native code and use gdb.

Beyond Java

Question:

Can I use LucidDB in a non-Java application?

Answer:

Maybe.

Compression Statistics

Question:

How can I tell how well compression is working?

Answer:

It can be difficult to determine this from the size of db.dat, since not all of the pages may be currently in use.

To find out how many pages are currently in use across all tables, across all page versions, execute this:

alter system deallocate old;
select counter_value from sys_root.dba_performance_counters where counter_name='DatabasePagesAllocated';

Multiply the result by 32KB (the size of one page).

To study compression effectiveness for a particular table, execute this:

alter system deallocate old;
analyze table t estimate statistics for all columns;
select index_name, pages from sys_boot.mgmt.page_counts_view where table_schem='YOUR_SCHEMA_NAME' and table_name='YOUR_TABLE_NAME';

Indexes with names starting with "SYS$CLUSTERED" represent the compressed column storage. Every table has a single "SYS$DELETION_INDEX" which tracks deleted rows. A table may have SYS$ indexes corresponding to PRIMARY KEY and UNIQUE constraint defintions. Any other indexes are named indexes you have created.

OpenSUSE Performance

Question:

Why is LucidDB so slow for short queries on OpenSUSE?

Answer:

See the discussion on the ext3 barrier setting in this thread, which contains a workaround.

Identifier Quoting

Question:

Why are the rules for this so confusing?

Answer:

LucidDB strictly adheres to SQL:2003 in this regard. When you don't quote identifiers, they are implicitly uppercased as part of parsing (e.g. yoUr_table becomes YOUR_TABLE). When you quote them, case is preserved (e.g. "yoUr_table" stays exactly like that). You must quote them if they are reserved words or contain special characters. For compound identifiers, the quotes go around each component, e.g. "yoUr_schema"."yoUr_table" not "yoUr_schema.yoUr_table").

One point of confusion arises from the SET SCHEMA command. The actual syntax is SET SCHEMA <character-expression>, not SET SCHEMA <identifier>. So, you have to write

SET SCHEMA '"yoUr_schema"'

not

SET SCHEMA "yoUr_schema"

If you have a schema named EXTRACTION_SCHEMA (all upper case), this works:

SET SCHEMA 'extraction_schema'

The reason this works is that the single quotes are delimiting the character literal (not the identifier). The parsing for this expression first evaluates the character expression, then interprets the result as a (possibly-quoted) identifier. This allows you to do things like

SET SCHEMA 'SCHEMA_FOR_' || CURRENT_USER

So if user FRANCISCO logs in, he'll get SCHEMA_FOR_FRANCISCO set as default after executing the expression above.