SQL Service (com.ibi.agents.XDSQLAgent)

Topics:

Syntax:

com.ibi.agents.XDSQLAgent

iIT Service Object:

SQL: SQL Agent

Description:

The SQL Service enables you to execute an arbitrary SQL statement, specified as a service configuration parameter. The service returns an EDA response document, as defined elsewhere. This service differs from the more complete RDBMS services and adapters in that it does not interrogate an iWay response document to obtain its SQL. Instead, it accepts the SQL as a configuration parameter and modifies the statement based on the current document or environment. The SQL expression can be modified during execution by iWay standard parameter expressions.

Using the SQL service instead of the RDBMS Adapter is preferred in cases where dynamically built queries are processed or when simple (non-looping) SQL statements need to be processed quickly. This service also provides conditional transactionality support, multi-part transactional support, advanced SQL language capabilities, and error-handing features.

Parameters:

Parameter

Description

SQL DML *

The SQL Data Manipulation Language (DML) statement, which can also evaluate an iFL expression to build an SQL statement dynamically. For example, inserting SREG(value) into the text of your query.

Evaluate SQL

If set to true, an attempt is made to treat the SQL as an expression. This may not be appropriate to all SQL. For example, quotes in evaluated strings must be escaped by doubling.

Output Format *

The format of the result set for a select operation:

  • Field. A format designed for efficient XPath.
  • Column. A format that is effective for transformations.
  • Simple. A format that contains minimal metadata, and produces simple JSON documents when converted.
  • Row. Deprecated. Returns the entire tuple as a single string. This format is not recommended for current JDBC operations. New configurations do not permit Row to be specified.

Examples are shown following this table.

Use JNDI *

If set to true, a connection from a data source accessed through JNDI is used. Otherwise, connect directly using JDBC driver manager. When set to true, the Pool Connections configuration parameter should not be used, and pooling is handled in the JNDI data source manager.

JNDI Name

JNDI name for the requested data source. To use an iWay JDBC provider, specify as jdbc/provider. This is required if you are using JNDI.

JNDI Factory

JNDI initial context factory class. Leave blank for default.

Transaction Isolation Level

Transaction isolation level to be set if possible.

These are the standard transaction Isolation levels defined in the ANSI/ISO SQL standard. For more information on these settings, see the DBMS, JDBC, or ODBC documentation. Based on standard terminology, the following transaction isolation levels can be selected:

  • As Is. Does not change the transaction isolation level. The default is accepted, as it is found on the connection.
  • Read Uncommitted. Allows reading of a record that may be rolled back later. Dirty reads, nonrepeatable reads, and phantom reads are possible.
  • Read Committed. Dirty reads are not possible, but nonrepeatable reads and phantom reads are possible.
  • Repeatable Read. Dirty reads and nonrepeatable reads are not possible, but phantom reads are possible.
  • Serializable. This is the highest level of isolation. Phantom reads are not possible. Other users are prevented from updating or inserting rows into the data set until the transaction is completed.

For more information on the terminology related to transaction isolation, see Transaction Isolation Terminology.

Pool Connections

If set to true, connections are pooled. The default value is false.

Applies to non-local transactions only. This pools connections within the flow itself. Connections can be reused across messages, depending on the commit strategy and the specifics of the database being used. If used with JNDI data sources, this local service pooling may inhibit optimum use of any JNDI data source pooling and connection management. In most cases, the Pool Connections parameter for this service should not be set when using JNDI data sources.

Pooled connections are identified by connection URL and user ID. The pooled connection is shared by all instances of this service either within the channel as a whole or within a single execution (worker) thread. To pool at the worker level, set the Need Commit parameter to true.

In local transactions, connections are always shared within the transaction.

Attempt Read Only

For select set JDBC read only flag. Some drivers cannot handle this optimization and can give security failures.

Connection Properties

Properties to apply to a connection.

Need Commit

Determines how commits should be performed on completion. Select one of the following options from the drop-down list:

  • Immediate. Issue a commit immediately upon completion of the Data Manipulation Language (DML) execution, regardless of whether or not the process is part of the local transaction. The connection is not pooled for commit purposes.
  • Transactional. If the process is running as a local transaction, the service will join that transaction. In this case a commit or rollback will be performed on the connection depending on the completion status of the process flow. If the process is not running in a local transaction, this defaults to the Immediate option.
  • Never. No commit is ever issued by the service for this statement. This is the default for a select statement. For a DML that requires a commit, the process flow is responsible for committing the connection by use of DML in another service. This gives the transaction complete control and responsibility for the commit or rollback. A commit or rollback is not executed by the use of the Commit service or by the transactional status of the process flow.
  • Force. Always issue a commit before closing the connection. Use only in non-transactional mode. Setting the Force option is required only for certain JDBC drivers that require a commit in order to close all resources. For example, some DB2 drivers will generate an error if a connection is closed without a commit. For other drivers, issuing a commit with no outstanding work can generate an error. Users are cautioned to only use the Force option if it is required by your driver and application design.
 

Note: Prior to iSM version 6.1.4, the Boolean values true (Immediate) and false (Transactional) were provided as options for this parameter. These internal settings continue to perform as in previous releases.

Users are cautioned that commits are issued on connections, rather than statements. Selecting the Never option delegates application responsibility for the connection to the application rather than the transaction manager of the process flow. Failure to execute an SQL service with never having a COMMIT WORK or ROLLBACK WORK (as appropriate for the specific database), will leave the connection uncommitted.

For DML-based statements with the Never option selected, you will require pooling enabled to insure that the TCL (Transaction Control Language) affects the proper connection. Pooling is automatic when using JNDI-based connections.

Timeout

Number of seconds this service will wait for an operation to complete. A value of 0 indicates no timeout check.

Max Rows

Maximum rows if query DML (0 is all).

Make Siblings

If set to true, sibling documents are created for each Max Rows group.

Sibling documents are full documents that are chained to the emitted document from the service. They can be accessed by use of an iterator, and will be emitted automatically if they exist for the final document that is output from the process flow.

Max Siblings

Maximum siblings of maximum rows (if siblings selected).

Emit No Empty

If set to true, empty documents are not emitted.

Issuance Strategy

Issuance strategy: through SQL construction or through SQL bind.

Reply Node Name

If entered, response is put at this fully qualified named node.

Base 64 if Needed

If set to true, all fields are checked for base64.

xLOB Handling

Determines how blobs and clobs should be handled. Select one of the following options from the drop-down list:

  • none (default)
  • external
  • inline

Want Generated Keys

If set to true, generated keys on a non-bound INSERT statement are returned if available from the database in the 'iway.getkey' register. The default value is false.

Call at EOS?

In a streaming environment, EOS (End of Stream) is the short message that is sent after the last document, which signifies the EOS. This parameter determines whether this service should be called for the EOS message. The default value is false.

JDBC Connection Properties

Data Source URL *

The URL to reach the data source.

JDBC Driver *

The JDBC driver to use.

User ID

Default user ID for the connection.

Password

Default password for the connection.

Passing Values to DML

When you create the SQL Data Manipulation Language (DML) statement, you can use named tokens to represent the values to be used in the statement. For example:

Select name, address from addresses where zip='?zipval' and state = '?stateval'

Then in the user properties, represent the values as name/value token pairs, as listed in the following table.

Name

Value

zipval

_xpath(/root/user/zip)

stateval

_sreg(state)

The SQL Service supports stored procedures containing a mix of input, output, and IN OUT parameters. To use a stored procedure with output parameters, you must describe each output parameter that is expected. The values returned by these parameters are stored in the response document generated by this service. The syntax for an output parameter is:

?$type(options).name

where:

options

Is optional.

name

Is optional.

The syntax for an IN OUT parameter is:

?#type(option).name

The type must be set to one of the constants listed in the following table.

JDBC Type

Constant

Option

BLOB

blob

CHAR

char

CLOB

clob

DATE

date

INTEGER

int

SQLXML

xml

TIMESTAMP

tstamp

VARCHAR

varchar

CURSOR (Oracle)

cursor

Numeric

numeric

scale

Decimal

decimal

scale

Input parameters follow the normal syntax for passing information into the DML.

For example, call a stored procedure named sp1 with a string input (perhaps a key) and receive an output of int, numeric, and a result set (Oracle):

{call ?int.result = sp1(?inkey,?$numeric(2).cost,?$cursor)}

This will return three output parameters (see formats below), indexed 1 (result), 2 (cost), and a result set.

A stored procedure that uses an input parameter might accept a partial key, and return the complete key along with the other data. For example:

{call sp1(?#varchar.inkey,?$numeric(2).cost,?$cursor)}

In either case, you would configure inkey as an input parameter as if it were simply ?inkey, and the value will also appear as an output parameter.

Examples of using out and IN OUT parameters with stored procedures are provided in Examples of Using the SQL Service With Stored Procedure Calls and Output Parameters.

The _qval() function can be used to obtain a null value. For example:

Insert into nametable (first,middle,last) values('?first',?middle,'?last')

Assuming that the first and last values cannot be null, use the following name/value token pairs to represent the values.

Name

Value

first

_sreg('firstname')

middle

_qval(_sreg('middle'),'\','empty')

last

_sreg('lastname')

If a middle name is not present, then the text NULL is used in the SQL. Otherwise, the quoted string of the middle name is used.

Edges:

The SQL service (and by extension the SQL object used in process flows) returns edges that can be used to analyze execution results at any required level of granularity. Any SQL service failure returns a fail_operation edge, but a duplicate or a missing result can also be used. At the finest granularity, any specific XOPEN code (or set of codes) can be tested.

The edges returned by the SQL service (com.ibi.agents.XDSQLAgent) are listed in the following table.

Edge

XOPEN

Description

success

00000

The operation was successful. For SELECT operations, success can be considered as the join of found and notfound edges.

found

00000

Rows were returned. Always issued in conjunction with the success edge. You may wire found or success edges depending on your application requirements.

notfound

00000

No rows were returned. You may wire notfound or success edges depending on your application requirements.

Note: The notfound edge indicates a successful operation, as distinct from the fail_notfound edge, which indicates that some components, such as a table or column, is not found in the database.

duplicate

3C000, 42S01, 42S11, 42S21

Some item is duplicated, for example, insert to an existing unique key.

fail_notfound

42S02, 42S12, 42S22

A component required in the DML, such as a table or column, was not found.

fail_operation

The SQL operation was not successful.

cancelled

A cancellation through a flow timeout.

fail_timeout

HYT00, HYT01, 40001

The operation times out on the channel.

fail_parse

Parameters cannot be parsed in iFL.

fail_syntax

4200, 22007

The SQL had syntax errors as reported by the DBMS.

fail_other

The SQL operation has failed for a reason that is not included in any other edge (for example, fail_notfound). You can use the sqlcode special register for further analysis.

<xopen code>

all

Specific XOPEN code. To use this edge, you must define your own custom edge in iIT Designer.

For more information on SQL Service edge returns sorted by XOPEN code, see Service Edge Returns Sorted by XOPEN Code.

The following table lists the Special Register (SREG) set:

SREG

Description

sqlcount

Number of rows returned for a SELECT operation, or the number of rows affected by an UPDATE or DELETE operation.

iway.genkey

Generated key associated with an INSERT operation, if appropriate to the database.

sqlcode

The XOPEN code for any unsuccessful operation.

Example:

An SQL Select statement returns a result set as an XML document. The service offers three formats:

Field format is designed for efficient XPath. Within each row, the fields are presented as children. For example:

<?xml version="1.0" encoding="ISO-8859-1" ?><iway>
 <response totalrows="3" totalupdate="0">
  <cncresult>
   <result format="field">
    <resultset rowcount="3">
     <colinfo>
      <col label="LNAME" length="8" offset="0" type="1" typename="char">LNAME</col>
      <col label="FNAME" length="8" offset="8" type="1" typename="char">FNAME</col>
      <col label="SSN" length="10" offset="16" type="2" typename="numeric">SSN</col>
      <col label="BDATE" length="11" offset="26" type="93"
      typename="tstamp">BDATE</col>
      <col label="SALARY" length="8" offset="37" type="2"
      typename="numeric">SALARY</col>
     </colinfo>
     <row>
      <LNAME type="1" typename="char">NARAYAN </LNAME>
      <FNAME type="1" typename="char">RAMESH  </FNAME>
      <SSN type="2" typename="numeric">666884444</SSN>
      <BDATE type="93" typename="tstamp">1952-09-15 00:00:00</BDATE>
      <SALARY type="2" typename="numeric">38000</SALARY>
     </row>
     <row>
      <LNAME type="1" typename="char">SMITH   </LNAME>
      <FNAME type="1" typename="char">JOHN    </FNAME>
      <SSN type="2" typename="numeric">123456789</SSN>
      <BDATE type="93" typename="tstamp">1955-01-09 00:00:00</BDATE>
      <SALARY type="2" typename="numeric">30000</SALARY>
     </row>
     <row>
      <LNAME type="1" typename="char">ZELAYA  </LNAME>
      <FNAME type="1" typename="char">ALICIA  </FNAME>
      <SSN type="2" typename="numeric">999887777</SSN>
      <BDATE type="93" typename="tstamp">1958-07-19 00:00:00</BDATE>
      <SALARY type="2" typename="numeric">25000</SALARY>
     </row>
    </resultset>
   </result>
  </cncresult>
  <timestamp>TIME-QA-QA QA:QA:QA</timestamp>
  <execstatus>0</execstatus>
 </response>
</iway>

Column format is often effective for transformations. For example:

<?xml version="1.0" encoding="ISO-8859-1" ?><iway>
 <response totalrows="3" totalupdate="0">
  <cncresult>
   <result format="column">
    <resultset rowcount="3">
     <colinfo>
      <col label="LNAME" length="8" offset="0" type="1" typename="char">LNAME</col>
      <col label="FNAME" length="8" offset="8" type="1" typename="char">FNAME</col>
      <col label="SSN" length="10" offset="16" type="2" typename="numeric">SSN</col>
      <col label="BDATE" length="11" offset="26" type="93"
      typename="tstamp">BDATE</col>
      <col label="SALARY" length="8" offset="37" type="2"
      typename="numeric">SALARY</col>
     </colinfo>
     <row>
      <column label="LNAME" name="LNAME" type="1" typename="char">NARAYAN </column>
      <column label="FNAME" name="FNAME" type="1" typename="char">RAMESH </column>
      <column label="SSN" name="SSN" type="2" typename="numeric">666884444</column>
      <column label="BDATE" name="BDATE" type="93"
      typename="tstamp">1952 09-15 00:00:00</column>
      <column label="SALARY" name="SALARY" type="2"
      typename="numeric">38000</column>
     </row>
     <row>
      <column label="LNAME" name="LNAME" type="1" typename="char">SMITH </column>
      <column label="FNAME" name="FNAME" type="1" typename="char">JOHN </column>
      <column label="SSN" name="SSN" type="2" typename="numeric">123456789</column>
      <column label="BDATE" name="BDATE" type="93"
      typename="tstamp">1955-01-09 00:00:00</column>
      <column label="SALARY" name="SALARY" type="2"
      typename="numeric">30000</column>
     </row>
     <row>
      <column label="LNAME" name="LNAME" type="1" typename="char">ZELAYA </column>
      <column label="FNAME" name="FNAME" type="1" typename="char">ALICIA </column>
      <column label="SSN" name="SSN" type="2" typename="numeric">999887777</column>
      <column label="BDATE" name="BDATE" type="93"
      typename="tstamp">1958-07-19 00:00:00</column>
      <column label="SALARY" name="SALARY" type="2"
      typename="numeric">25000</column>
     </row>
    </resultset>
   </result>
  </cncresult>
  <timestamp>TIME-QA-QA QA:QA:QA</timestamp>
  <execstatus>0</execstatus>
 </response>
</iway>

Row format was originally designed for extremely fast retrieval, but your application is responsible for extracting information for the rows through string manipulation. This format matches the format returned by the EDA API of the Full Function Server, and should only be used with that database. iWay Software does not recommend this format for most general applications.

<?xml version="1.0" encoding="ISO-8859-1" ?><iway>
 <response totalrows="3" totalupdate="0">
  <cncresult>
   <result format="std">
    <resultset rowcount="3">
     <colinfo>
      <col label="LNAME" length="8" nullable="1" offset="0" type="1"
      typename="char">LNAME</col>
      <col label="FNAME" length="8" nullable="1" offset="8" type="1"
      typename="char">FNAME</col>
      <col label="SSN" length="10" offset="16" type="2"
      typename="numeric">SSN</col>
      <col label="BDATE" length="11" nullable="1" offset="26" type="93"
      typename="tstamp">BDATE</col>
      <col label="SALARY" length="8" offset="37" type="2"
      typename="numeric">SALARY</col>
     </colinfo>
     <row>NARAYAN RAMESH  666884444 1952-09-15 00:00:0038000   </row>
     <row>SMITH   JOHN    123456789 1955-01-09 00:00:0030000   </row>
     <row>ZELAYA  ALICIA  999887777 1958-07-19 00:00:0025000   </row>
    </resultset>
   </result>
  </cncresult>
  <timestamp>TIME-QA-QA QA:QA:QA</timestamp>
  <execstatus>0</execstatus>
 </response>
</iway>

Examples of Using the SQL Service With Stored Procedure Calls and Output Parameters

Topics:

When using the SQL Service to execute any stored procedure with parameters, iWay Software recommends that the service be configured to use the bind issuance strategy. However, when a procedure has output parameters, the bind strategy must be used.

Calling a Stored Procedure With Input and Output Parameters

The following Oracle procedure accepts a varchar as input and returns a varchar and a date as output values:

Create or Replace Procedure HELLO_TODAY 
( P_NAME IN VARCHAR2,  
  P_HELLO OUT VARCHAR2,
  P_TODAY OUT DATE
) 
AS 
BEGIN
  SELECT CURRENT_DATE INTO P_TODAY FROM DUAL;
  P_HELLO := 'Hello ' || P_NAME;
END HELLO_TODAY;

To execute the procedure, the SQL DML is specified as:

{call HELLO_TODAY(?name, ?$varchar, ?$date)}

The following table describes how the three parameters are specified:

Parameter

Description

?name

An input parameter, to which the value of the user parameter name, evaluated at execution time will be bound.

?varchar

An output parameter of type varchar. Note that no name is specified. In the response document, the value of this parameter can be identified from its index.

?date

An output parameter of type date. Again, no name is specified, but the resulting value can be identified from the index of the parameter.

If the procedure executes successfully, the SQL Service will generate the following type of response document:

<iway>
   <response totalrows="0" totalupdate="-1">
      <cncresult>
         <result format="column">
            <outparm index="2" 
                     name="parm2" 
                     type="varchar" 
                     jdbctype="12">Hello Stephen</outparm>
            <outparm index="3" 
                     name="parm3" 
                     type="date" 
                     jdbctype="91">2013-03-08 10:46:10.0</outparm>
         </result>
      </cncresult>
      <timestamp>2013-03-08T15:46:11Z</timestamp>
      <execstatus>0</execstatus>
   </response>
</iway>

Note: The value of the index attribute on the outparm elements refers to the index of the parameter relative to all parameters, not just output.

Calling a Stored Procedure With an IN OUT Parameter

The following simple procedure uses the same parameter for both input and output:

Create or Replace Procedure INOUT_HELLO (P_HELLO_NAME IN OUT VARCHAR2) 
AS 
BEGIN
  P_HELLO_NAME := 'Hello ' || P_HELLO_NAME;
END INOUT_HELLO;

The SQL DML for the service can be specified as:

{call INOUT_HELLO(?#varchar.name)}

The IN OUT parameter is specified as:

Parameter

Description

??#varchar.name

This time, the ?# prefix is used to indicate that this is an IN OUT parameter of type varchar. For input, you will bind the value of the user parameter name and you will also use name to identify the output of this parameter in the response document. An IN OUT parameter must be defined with a name.

If the procedure executes successfully, the SQL Service will generate the following type of response document:

<?xml version="1.0" encoding="ISO-8859-1" ?>
<iway>
   <response totalrows="0" totalupdate="-2">
      <cncresult>
         <result format="column">
            <outparm index="1" 
                     name="name" 
                     type="varchar" 
                     jdbctype="12">Hello Stephen</outparm>
         </result>
      </cncresult>
      <timestamp>2013-03-08T19:49:04Z</timestamp>
      <execstatus>0</execstatus>
   </response>
</iway>

Calling an Oracle Procedure With an Input and a Cursor Output Parameter

The following procedure returns data from the iWay Business Activity Monitor (BAM) database, based on a pattern passed as an input parameter:

Create or Replace Procedure BAM_ACTIVITY_LIST 
( P_TID IN VARCHAR2  
, P_RC OUT SYS_REFCURSOR 
) 
AS 
BEGIN
 open P_RC for
  select recordkey, recordtype, tstamp from BAM_Activity 
   where TID like P_TID;
END BAM_ACTIVITY_LIST;

To invoke the procedure with the SQL Service, the SQL DML parameter is set as:

{call BAM_ACTIVITY_LIST(?mytid, ?$cursor.tidresult)}

The following table describes how the two parameters are specified:

Parameter

Description

?mytid

An input parameter, to which the value of the user parameter mytid, evaluated at execution time, will be bound.

??$cursor.tidresult

An output parameter that will have the cursor datatype and be identified in the response document as tidresult.

If the service succeeds in executing the procedure and is configured to output its result in column format, then the service will generate the following type of response document:

<?xml version="1.0" encoding="ISO-8859-1" ?>
<iway>
  <response totalrows="28" totalupdate="-1">
    <cncresult>
      <result format="column">
        <outparm index="2" name="tidresult" type="cursor" jdbctype="-10">
          <resultset rowcount="28">
            <colinfo>
              <col length="32" 
                   offset="0" type="1" 
                   typename="char" 
                   label="RECORDKEY">RECORDKEY</col>
              <col length="22" 
                   offset="32" 
                   type="2" 
                   typename="numeric" 
                   label="RECORDTYPE">RECORDTYPE</col>
              <col length="11" 
                   offset="54" 
                   type="93" 
                   typename="tstamp" 
                   label="TSTAMP">TSTAMP</col>
            </colinfo>
            <row>
              <column name="RECORDKEY" 
                      label="RECORDKEY" 
                      type="1" 
                      typename="char">5dc36681b5f742b09d7ebbb10a79ae7e</column>
              <column name="RECORDTYPE" 
                      label="RECORDTYPE" 
                      type="2" 
                      typename="numeric">101</column>
              <column name="TSTAMP" 
                      label="TSTAMP" 
                      type="93" 
                      typename="tstamp">2012-5-10.14.50. 40. 900000000</column>
            </row>
          </resultset>
        </outparm>
      </result>
    </cncresult>
<timestamp>2013-03-08T14:40:17Z</timestamp>
    <execstatus>0</execstatus>
  </response>
</iway>

Note: The resultset is formatted in exactly the same way as if it had been returned from a SELECT statement, except that it becomes a child of the outparm element. Also, note that the name attribute of the output element holds the out parameter name as specified in the SQL DML.