Topics: |
Reference: |
The Ordered Queue listener is used to manage an ordered queue. In this section, the term queue refers to the entire set of messages that are awaiting transfer to a subchannel.
The ordered queue is a collection of queuelets. Each queuelet collects the messages of one group in the order they were received. A new queuelet is created automatically when a message comes in with a key that has no associated queuelet. A queuelet is automatically deleted when it becomes empty, that is, when all the messages in that group have completed execution. Messages are deleted from the queuelet after their execution is completed. This ensures that messages cannot be lost when the queue is configured to be persistent, and maintains the ordering should a message arrive for the group while execution of the last message is proceeding.
This section will address differences in configuration between the simpler internal queue and the ordered queue. For more information on how the ordered queue system works for the ordering of messages, see Introducing the Ordered Queue Facility.
To configure an Ordered Queue listener, you must create a channel for your application project using the Channel Builder in iWay Integration Tools (iIT) and select Ordered Queue as the listener type for your inlet. For more information, see Configuring a Listener Using iWay Integration Tools.
For a complete description of the configuration parameters that are available for the Ordered Queue listener, see Ordered Queue Listener Configuration Parameters.
For a complete description of the Ordered Queue listener Special Registers (SREGs), see Ordered Queue Listener Special Registers.
Pended messages are always executed in the proper order within the group. The implication is that it is the group itself that is pended rather than the individual message. Consistent with the normal server rules, if the pended message times out or fails, then it is considered to be complete and the next message in the queuelet is made available for dispatch.
In immediate mode, the messages are available for dispatch to the subchannel (process flow) immediately upon their arrival at the listener.
In batching mode, a new message is put in its queuelet. However, the queuelet simply collects the messages without making them available for dispatch. When the end of group signal is received for that group, the queuelet becomes available for execution, just like in immediate mode.
A delete group message can be sent to delete an unended group. A delete message cannot be used for an ended group.
In persistent mode, the end of group signal is pended along with the messages, thus instructing the channel to correctly end the group when reloading it from safe store. An unended group is reloaded as an unended group. Application logic must decide whether to end that group or delete it.
Control signals, such as delete group, are sent through the Ordered emit service (com.ibi.agents.XDOrderedEmitAgent), which is configured using the Queue Out object in the iWay Process Flow Designer and selecting the iWay queue (ordered) - send a message action. Available signals are listed and described in the following table:
Signal |
Description |
---|---|
Document {doc} |
Current document message is to be enqueued for its group key. |
Delete {del} |
Batch queue is to be deleted. |
End Signal {end} |
Final message has been enqueued for this group, and the batch queue is to be made available for dispatching. |
Keep Alive {keepalive} |
Resets the timeout period for a batch queue. |
Last Document {docend} |
Effects a document and end operation. Signals that this message is the final message and the queue is to be made available for dispatching. |
The following table lists and describes the general set of special registers (SREGs):
Special Register |
Description |
---|---|
endofgroup |
Set to true when the last message of a batching queuelet is dispatched to the process flow. This register is set to true for this final message. Emitted signals, such as the end of group signal are not dispatched to the associated process flow. |
msgkey |
Group key associated with this message. |
sortkey |
Sort key for non-immediate processing. Not applicable for non-sorted immediate processing. |
The ordered listener has a parameter to define the timeout before a batching group must be closed. This is the maximum time between the last message of a group and receipt of the end of the group signal. The timeout does not apply to queues in immediate mode. The Expired Group Event Flow parameter holds the name of a process flow to run when the group timeout expires. If no process flow is specified, then the server writes an error message in the log. The messages are enqueued to the dead letter destination (assuming one is configured) and the group is deleted. If a process flow is specified, then the server runs that process flow. The process flow can send new messages, the delete group message, the end of group signal, or the keep alive signal to reset the timer for the specified group. If the process flow returns a success condition, then the server considers the condition to be handled, and no further action is taken. If the process flow returns a condition other than success, then the server reverts to the default behavior as if there were no process flow.
The expired group flow receives a signal message in the event of the timeout expiration. This flow can elect to emit a keep alive or a delete signal message to the group.
<signal errors="0" protocol="ordered" timestamp="2018-10-24T17:56:30.730Z" type="expired" version="2"> <parms count="2"> <parm name="queue">ordered</parm> <parm name="key">patient1</parm> </parms> </signal>
The type of the signal document is expired and specifies the queue and group names, which in this example are ordered and patient1.
If the channel in immediate mode is stopped, then the messages behave as defined for the Internal Queue listener.
When the server starts, it searches for persisted messages awaiting execution. The queuelets are reconstructed to their status prior to the when the server was stopped. Immediate mode and ended batch mode queuelets begin dispatching messages immediately, while unended batch mode queuelets wait for an end of group signal.
Application designers are reminded that the timeouts applied to batch mode queues are based on wall clock time. This can result in the execution of timeout flows, and the application logic should take this into account.
The following table lists and describes parameters for the Ordered Queue listener.
Note: Parameters that are common to queue listeners are described in Listener Configuration Parameters.
Property |
Description |
---|---|
Name of Ordered Queue (required) |
Name of the ordered queue that is used to identify the ordered message listener destination. |
Queuing Mode |
Determines how received messages are handled. Available modes include:
The default value is immediate. |
Sorting Mode |
Applies to batch queues only, and only within the batch itself. Available options include:
The default value is Chronological. |
Group Timeout |
Indicates the time that a batching queue can remain unavailable for dispatching following the receipt of the latest message. In effect, this is an inactivity timeout. The default value is 300 seconds. |
Expired Group Flow |
Name of a published process flow that receives control if an unended group times out. |
Persistent |
If enabled, messages are persisted. Persistent messages are held in the safestore until completion and can be recovered if the server is restarted. Select one of the following options from the drop-down list:
The default value is none {false}. The rdbms option enables the application to persist messages to a remote database location where, in the event of a failover situation, a secondary server running remotely can continue processing persisted messages without any business interruption. To use RDBMS persistence, you must create the iway_queues table in the JDBC provider, using the DDL script in the etc/setup directory. A hot backup channel can be configured to use the same RDBMS safestore. |
Safestore Location |
If persistent, this is the location in the file system to which documents are safestored. |
Compress Persistent |
If the queue is persistent, the documents written to the safestore can be compressed. Select one of the following options from the drop-down list:
The default option is to not compress (none). |
Low Mark |
If the size of the named queue falls below this value, then the named listeners that are specified in the Control List parameter are sent an activate message. The low mark value applies to the queue as a whole, and not to the individual quelelets. The default value is 0. |
High Mark |
If the size of the named queue goes above this value, then a passivate message is sent to the listeners that are specified in the Control List parameter. The high mark value applies to the queue as a whole, and not to the individual quelelets. The default value is 0. |
Control List |
If a high or low mark value is crossed, the appropriate message is sent to each listed listener. For example, if two listeners LISA and LISB are feeding the internal queue, listing them as LISA,LISB will cause each to receive the appropriate message. |
Inhibit Add |
If set, the queue will not accept new messages when its size reaches the high mark, and will resume accepting messages when the number of messages on the queue reaches the low mark. The effect of this inhibition can cascade through the application, controlling overall performance. The default value is false. |
External Mark |
When set to a number greater than 0, this causes the message to be stored in memory up to the specified number. When the queue grows larger than the specified number, the message is stored on disk. This is equivalent to splitting the memory and performance optimization for the messages in the queue, and is effective for large lists that may fill memory. The default value is 0. |
Duration |
Maximum time in seconds (allows xhxxmxxs format) that a document can remain in this channel, starting from when the document is added to the queue, including pends and retries. The default value is 24 hours. |
Support Pending |
If set, messages can be pended for later execution if the process flow calls for a Fail/Pending operation. Pending messages persist for a specified time and are retried at a specific interval. You might use a pending operation in the event that a message cannot be processed because an external resource is currently unavailable. The default value is false. |
Retry Interval |
Determines the interval (in seconds) between retrying pending requests. The default value is 600 seconds (10 minutes). |
The following table lists and describes the Special Registers (SREGs) available on the Ordered Queue listener. These values can be used in the application for message routing and processing.
Name |
Source |
Level |
Type |
Description |
---|---|---|---|---|
endofgroup |
Listener |
Document |
Boolean |
Indicates the last message of a queuelet group. |
iway.channel |
Listener |
System |
String |
Full name of the channel (may include channelname.inlet.listener). |
iway.channelname |
Listener |
System |
String |
Channel name portion of the name from the full channel name of channelname.inlet.listener. |
iway.inletname |
Listener |
System |
String |
Inlet name portion of the name from the full channel name of channelname.inletname.listener |
iway.listener |
Listener |
System |
String |
Name of the listener. |
iway.pid |
System |
System |
String |
Process ID of the server, if available. |
iway.serverfullhost |
System |
System |
String |
Full host name of the server (includes domain). |
iway.serverhost |
System |
System |
String |
Host name of the server. |
iwayconfig |
System |
System |
String |
Current active configuration name. |
iwayhome |
System |
System |
String |
Base at which the server is loaded. |
iwayversion |
System |
System |
String |
Release version of the server. |
iwayworkdir |
System |
System |
String |
Path to the base of the current configuration. |
msgkey |
Listener |
Document |
String |
Batch key being dispatched. |
msgsize |
Listener |
Document |
Integer |
Physical length of the message payload. |
name |
Listener |
System |
String |
Assigned name of the master (same as iway.channel). |
protocol |
Listener |
System |
String |
Protocol on which the message was received. |
sortkey |
Listener |
Document |
String |
Values being sorted for ordered dispatch, if applicable. |