TorQ Release v3.3 – kdb+ 3.6

Blog kdb+ 14 May 2018

Data Intellect

We are delighted to announce release v3.3 of TorQ, the latest installment of our kdb+ framework; which can be found here. The TorQ v3.3 release has upgraded to run with kdb+ 3.6, whilst maintaining backward compatibility with prior versions.  The main changes are outlined below.

Concurrent Synchronous Requests

As part of this update, the gateway in TorQ supports the deferred response feature, and will automatically be in this mode for kdb+ v3.6. The deferred response feature allows the gateway to suspend the processing of synchronous messages, and process other messages before it sends the response back to the client. This means that the gateway no longer must be either synchronous or asynchronous. A response formatter has been implemented in the gateway to specify how results are packaged for synchronous messages, giving greater flexibility when handling errors.

An example of this would be client A sending a long running synchronous query:

q).z.p
2018.03.22D17:44:40.367748000
q)h(`.gw.syncexec;"system\"sleep 10\";.z.p";`hdb)
2018.03.22D17:44:52.500426000

Client B can send a query afterwards and it will be executed before the initial query result is returned to Client A, as the gateway has put this original query on hold.

q).z.p
2018.03.22D17:44:44.725457000
q)h(`.gw.syncexec;".z.p";`rdb)
2018.03.22D17:44:45.888621000

Start and Stop Script (torq.sh)

This script is linux only, and can start or stop processes separately, in a batch or all at once.  Process configuration is now taken from the process.csv file which means that processes only have to be defined in one place (previously adding a new process would require an amendment to the start script, stop script and process.csv).

Before a process is started or stopped the script will check that the process is not already running before attempting to start or stop a process.  The default process file found in the TorQ-Finance-Starter-Pack can be found here. To add a new process, simply append the default startup configuration in the process.csv and use the start line command shown below to start or stop the new process.

An example of a start line to start an rdb, hdb and tickerplant process is shown below.

$ ./torq.sh start rdb1 hdb1 tickerplant1
15:42:00 | Starting rdb1...
15:42:00 | hdb1 already running
15:42:00 | Starting tickerplant1...
$ ./torq.sh stop all
15:46:19 | Shutting down hdb1...
15:46:19 | Shutting down hdb2...

A status summary table of all the processes is printed to screen, providing information on the time the process was checked, process name, status, port number and PID.

$ ./torq.sh summary
STATUS PROCESS STATUS PORT PID
11:33:59 discovery1 up 41001 14426
11:33:59 tickerplant1 down

The debug command line parameter can be appended to the start line straight from torq.sh to start a process in debug mode. Note it is only possible to start one process at a time in debug mode.

$ ./torq.sh tickerplant1 -debug

To add/override the default values in the g, T, w, or extras column the extras flag can be used in this script.

$ ./torq.sh start rdb1 -extras -T 60 -w 4000
$ ./torq.sh start sort1 -extras -s -3

Initialisation Function

An optional initialisation function call has been added as .proc.init which can be overridden in the file that is loaded and only invoked after the script is loaded and connections are established.

If you are working with kdb+ TorQ, we welcome your feedback for future development so please get in touch!

Share this:

LET'S CHAT ABOUT YOUR PROJECT.

GET IN TOUCH