From Eigenpedia

Contents 1 Overview 2 Requirements 3 Interface 4 Implementations 5 Old Work 5.1 Overview 5.2 Current implementation 5.3 Proposed implementation 5.3.1 Rel construction code 5.3.2 IterCalcRel implementation 5.3.3 Runtime libraries 5.3.4 Sample extension (user level configuration) 5.4 Previous review comments

Overview

This page discusses error handling, with a focus on Java calculator streams. This allows us to deal with errors such as arithmetic overflow or casting errors, and in some cases, not null constraints. Errors from other streams will be handled as required, and can be processed by a common error handling framework. This work supports LucidDb requirements, but does not change Farrago's existing behavior or hinder other extensions. Because error handling can vary so widely, this spec is fairly abstract, containing high level interfaces. The bulk of implementation actually occurs at a lower level.

Requirements

A Java calculator stream (IterCalcRel) processes one row at a time. The stream reads a row of input, calculates output columns sequentially, then writes a row of output. In this context, error handling consists of catching RuntimeExceptions and acting upon the exceptions. Two configuration points are whether to catch exceptions and when they are caught, what to do with them.

Farrago does not catch exceptions. For extensions, it is usually sufficient to catch exceptions on a row by row basis. Catching exceptions on a column basis has not been required and would probably be slow.

When exceptions are caught, one Farrago extension might send them to log files. Another might send exceptions to an error stream for further processing. Any two Java calc stream can have the same target or different targets.

Interface

By themselves calculator streams are not configured to handle errors. Calculator streams arise from a combination of filters and projections, through many rules. Rather than consider error configuration in every calculator rule, we consider error configuration once, with a Java calc stream decoration rule. A Farrago extension with rule set A would generate a different decoration than a Farrago extension with rule set B.



Note: To avoid doubling the optimization search space, a system based on a Volcano optimizer could configure errors during the calc to Java calc stage.

For simplicity, Java calc streams are decorated with a string TAG. Tags should be logical and are mainly used to differentiate between possible targets (log file A vs. log file B or stream A vs. stream B.)

All errors are published to a connection variable (better known as the runtime context). The actual action to be taken depends on how the context is set up and how it interprets tags. The context arises from a FarragoSessionPersonality and can be extended through that interface.

If a Java stream has been decorated with a tag, it generates the following Java code to support error handling. (PLACE_HOLDERS are shown rather than actual identifiers.)

SyntheticObject INPUT_ROW = ...;
try {
    COLUMN_INDEX = 1;
    // calculate first output column ...
    COLUMN_INDEX++;
    // calculate second output column ...
    // ...
} catch (RuntimeException ex) {
    // TAG is a hardcoded string, mapped to an error configuration
    CONNECTION.handleRowError(ex, TAG, COLUMN_INDEX, INPUT_ROW);
    // NOTE: if error could not be handled due to NoDataReason
    // keep track of error and handle it next time
}

Connections (a.k.a. runtime contexts) have a new method for handling row errors. If possible, we want to find the output column for which the error occured.

Implementations

Farrago. The default Farrago implementation does not supply IterCalcRel decorations. When error handling is activated via IterCalcRel.setAbortOnError(false), a null tag is passed to the runtime context. The only action taken is to trace via a statement tracer.

LucidDb. In short, this database is configured via session parameters and records errors in specific log files. LucidDbSessionPersonality extends the default personality and returns a LucidDbRuntimeContext to handle errors. LucidDb is optimized by a heuristic planner and tags are assigned late in optimization by rules that operate on IterCalcRel. Based on a combination of session parameters and tags, a log file is chosen for logging. Individual log file writers are wrapped with a quota object to limit the number of errors.

Error stream. SWZ describes a scheme where errors are sent to an error stream. The target stream is selected via an extension to the SQL language. Presumably, during the Sql2Rel translation phase, the error target is saved as part of the statement. A rule is later able to use this error target to decorate an IterCalcRel. Presumably the runtime context will be equipped an adapter which can send errors detected to the desired output stream.

Note. In supporting both log files and streams, we have at least two main options. Here we chose to handle errors one by one and require a converter to translate them into streams. Alternatively we could have chosen to allow a Java stream to generate multiple outputs, with one output being the error stream. Such an approach would be flexible and beneficial to Java in the long run, but would require a significant change to the Java runtime infrastructure.

---

Old Work

Overview

This page covers runtime error handling in generated Java calculator code.

A Farrago statement usually fails when an error is detected. This includes cancelling a million row statement if the millionth row fails. Extensions of Farrago may want to refine their error handling behavior. For example, in the case of large or long running statements, we may want to recover from errors and track them.

select cast (EMP_CODE as int) from EMPS;
 

EMP_CODE
'002'
'007'
'MI5'
'009'

could return:

2
7
9

along with tracking the failed input "MI5".

Current implementation

Currently, error handling depends on a global server parameter. This parameter is not meant to change while the server is running. (The calculator portion of statements is compiled to Java code through an IterCalcRel.)

class IterCalcRel
{
    static boolean abortOnError;
    ...
    if (!abortOnError)
        // generate exception handling code
}

This generated code handles exceptions:

try {
    /* calculator statements */
}
catch (RuntimeException ex) {
    EigenbaseTrace.getStatementTracer().log(...)
}

The statement tracer is a Java class-specific statement tracer:

public static Logger getStatementTracer()
{
    return Logger.getLogger(OJPreparingStmt.class.getName());
}

Proposed implementation

Error handling will be standardized as follows:

• IterCalcRel generates code to publish all errors to error handlers
• Error handlers are obtained through the FarragoRuntimeContext

Various aspects of error handling, such as whether to fail after 5 errors, or whether to sends results to an error table, are encapsulated in an error handler. The runtime context may be configured to return different handlers, depending on:

• Session parameters, such as logical job id. For example, a job may have an id of "LoadOpportunity" or "S".
• Within a job, each instance of IterCalcRel may be given a tag

For example, in the context of a particular statement, a logical job id has been defined. None of the IterCalcRel instances have been given a tag. As a result, all instances of IterCalcRel would rely on the same error handler. All errors related to the statement are piped to the same place. In Farrago, the default error handler re-throws exceptions it receives (to abort on error).

Error handlers can take various actions. For example:

• An error handler may choose to pipe its results to a specific table, such as "S errors". The (psuedo) table may be registered and recognized for later queries "select * from S errors" including one with transformations on the columns "select func(col) from S errors".

(Does the correct producer/consumer interaction require registering the error stream as a consumer of the original stream?)

Rel construction code

The Farrago planner tags an IterCalcRel for rel-specific error handling as follows:

String tag = ...;
calcRel.setErrorHandlerTag(tag);

By default, CalcRels will not receive a tag when they are generated from random projections or filters. CalcRels cannot be merged when their tags differ.

As specified, the tags are open to interpretation. One Farrago extension may use them as a key from which to lookup an error log table. Another Farrago extension may use them to decide whether an error is recoverable.

IterCalcRel implementation

IterCalcRel generates the following Java calculator code

try {
    // calculator statements
} catch (RunTimeException ex) {
    ErrorHandler handler = runtimeContext.getErrorHandler(tag);
    RelDataType rowType = runtimeContext.getRowType(relId);
    Object status = handler.handle(ex, rowType, syntheticObject);
    if (status instanceof TupleIter.NoDataReason) {
        return status;
    }
}

(Would an array of fields be better than a single syntheticObject?)

Runtime libraries

The Farrago runtime context is configured by session parameters. It in turn configures and returns error handlers.

class FarragoRuntimeContext {
    /**
     * Returns an environment specific error handler. The default
     * implementation returns a handler that aborts on failure.
     *
     * @param relTag a rel specific tag; may be null
     */
    ErrorHandler getErrorHandler(String relTag);
}

Error handlers take actions upon receiving exceptions.

class ErrorHandler {
    /**
     * Processes a runtime exception
     *
     * @param ex  the exception encountered
     * @param tuple the row currently being processed
     *
     * @return status code, may be TupleIter.NoDataReason to
     * signal that no more rows should be processed until consumers
     * have proceeded, may be null
     */
    Object  handle(RunTimeException ex, TupleData tuple)
}

JVS: That should be FarragoSessionRuntimeErrorHandler, right? This is going to be exposed at the FarragoSessionRuntimeContext level, so macker will insist. Also, what is the actual type of TupleData? This method definition doesn't match the example invocation above, which takes both rowType and syntheticObject.

Sample extension (user level configuration)

An ETL process may be concerned with allowing nightly data loads to proceed in the absence of exceptional conditions (for example, when most rows are errors). Logs are helpful to use as a diagnostic feature, and preserving the data is not as important as viewing it in an intelligible way. (All error logs are stored in a directory for easy access, for a specific job, all errors are stored in the same file.)

Parameters such as log directory, job id, and etl process id, are set using sql. Statements on the same connection will inherit these attributes:

call sys_root.set_session_log_dir('/home/luciddb/log');
call sys_root.set_session_etl_proc_id('14612');
call sys_root.set_session_job_id('LoadOpportunity');
...
insert into staging_table select * from external table
abort after 100 errors;

JVS: Rather than mucking up the INSERT syntax with extra options like ABORT AFTER 100 ERRORS, let's just set these via new procedures such as sys_root.set_session_dml_err_abort_threshold.

Internally, a table is maintained to keep track of all error logs. Note that a single log file is generated for each logical job.

Date	Link	Statement
06-13-2001	LoadOpportunity14612	insert into staging_table ...

A user interface is able to display the contents of this table. When a user clicks on a link, the interface opens the associated log file. All error logs are stored to a single directory (for access by a single flatfile server.) Errors logs are named as follows (sorted by job id, then etl process id):

server_log_directory/<job_id><etl_proc_id>.log

Log files contain the following information:

Tag	Exception	Row
Flatfile2	Unable to convert MI5 to integer	MI5, Female, 32, UK

Note that the "row" column of a particular log file contains entries for multiple rels. Therefore its contents are not in a consistent format. This makes for easier viewing, but perhaps not for recovery. (One cannot do the following:)

insert into recovery_table
select col3, col4, col5 from errorlog;
 
insert into staging_table
select clean_phone(col3), col4, col5 from recovery_table;

JVS: I guess someone must not have liked my multi-file-in-a-subdirectory idea? Was that Tai?

Previous review comments

SWZ: Note that error handling is not currently turned on or off per-statement. Flipping abortOnError back and forth while Farrago (or an extension project) is running will probably cause weird behavior. Some of the code below implies that abortOnError isn't going away -- if that's actually the case maybe its value should be loaded from the session personality?

JVS: This makes it sound like IterCalcRel is where the configurability will be, but actually we'll be putting that in FarragoRuntimeContext, right? IterCalcRel will just publish the error events, and what gets done with them depends on how the logger created by the runtime context is configured.

JVS: Note that "type" here is a forward reference to something you define below, which is a little confusing. Calling the method setExceptionLogger makes it sound like it's going to take a java.util.logging.Logger, which isn't the case. Any attributes set on the rel need to be something that can be part of its digest (and therefore will show up in EXPLAIN PLAN). Maybe we can combine these two (i.e. if type=NONE then no recovery).

SWZ: Another concern is that, by default, multiple CalcRels with different recovery flags can be consolidated into a single CalcRel by the planner. As I mention below we're interested in something like diverting errors into an error table. We'd like to configure the error table either as part of a select statement (imagine "select * from s errors to s_errors") and/or as part of a view ("create view v as ... errors to v_errors"). Thus, "select someOperator(v.someCol) from v errors to my_errors" would divert different errors to different error tables. For this to work, we'd need the sql to rel conversion to configure at least some of the recovery information and planner rules would need to know not to combine rels with different recovery information.

SWZ: Doesn't the recovery flag subsume abortOnError? (Maybe this is just showing that abortOnError will live on until no one's using it?)

JVS: Do we actually need an EigenbaseLogger class? See "Subclassing Information" in Javadoc for java.util.logging.Logger. If we provide a runtime library method which converts tuple data into a LogRecord, then we could just use Logger.log(LogRecord). The format string can be generated as part of codegen and burned into the generated Java class.

SWZ