kdb+ Online Training Updated- Architecture

Jonny Press data capture, kdb, kdb+, TorQ, training Leave a Comment

We recorded our recent kdb+ Architecture Workshop in London, and we’ve added the video and slide deck as a bonus module on our kdb+ Bootcamp Online Training Course.  Above is a snapshot – a three hour in-depth workshop in 15 seconds.  Topics covered include: the basics of data capture extending kdb+ tick and alternative strategies scaling throughput versus latency removing …

Jonny Presskdb+ Online Training Updated- Architecture

Faster Recovery with kdb+ tick

Jonathon McMurray data capture, kdb, kdb+, TorQ Leave a Comment

The standard tickerplant (tick.q) from kdb+tick receives updates from feeds and pushes these updates to subscribing processes, as well as writing them to the log file. The tickerplant (TP) will count the number of messages received from the feed and written to the log file (in a variable .u.j) as well as the number of messages pushed to subscribers (in …

Jonathon McMurrayFaster Recovery with kdb+ tick

Free of Charge kdb+ Architecture Workshop NYC

AquaQ Admin data capture, datablog, kdb, kdb+, News, TorQ, training Leave a Comment

We were discussing our launch party in NYC and were kicking around ideas of what to do before it.  Some of us wanted to go to the Metropolitan Museum of Art; some of us wanted to go for a long lunch and maybe a stroll through Central Park; some of us wanted to run a kdb+ Architecture Workshop. So here we are. …

AquaQ AdminFree of Charge kdb+ Architecture Workshop NYC

TorQ User Group

AquaQ Admin data capture, datablog, kdb, kdb+, TorQ Leave a Comment

We have recently created a TorQ user group, for questions and discussions on TorQ (our kdb+ framework).  We will also be using it for more general data capture architecture considerations, and we’ll be publishing coding challenges every two weeks. We hope it will become a useful resource!

AquaQ AdminTorQ User Group

End-of-Day Parallel Sorting in TorQ

AquaQ Admin data capture, datablog, kdb, kdb+, TorQ Leave a Comment

We’ve written some blog posts previously on TorQ End-of-Day and more efficient methods for structuring temporary data on disk.  Until now TorQ has handled the end-of-day sorting or merging of tables sequentially, processing one table at a time.  Additional functionality has been added in the latest release of TorQ (v2.6.2) which will enable the parallel sorting of tables. This will allow …

AquaQ AdminEnd-of-Day Parallel Sorting in TorQ

kdb+ 3.4: Broadcast Publish

Glen Smith data capture, datablog, kdb, kdb+, TorQ Leave a Comment

kdb+ v3.4 introduces a new broadcast feature that reduces the work done when publishing messages to multiple subscribers.  Whenever a message is sent from one process to another via IPC (Inter Process Communication), the message is serialised before being sent and then deserialised when received.  Under normal IPC, if the same message is sent to multiple processes, it is serialised separately …

Glen Smithkdb+ 3.4: Broadcast Publish

TorQ v2.6 – kdb+ v3.4 Functionality Rolled In

Glen Smith data capture, kdb, kdb+, TorQ Leave a Comment

The latest release of TorQ can be found here. For this release we focused on two areas : incorporating new features from kdb+ v3.4 and improving our own processes. Unix Domain Sockets and SSL/TLS Our IPC library can now make use of Unix domain sockets (UDS) and SSL/TLS which have been added to kdb+ v3.4. UDS are not backwards compatible so …

Glen SmithTorQ v2.6 – kdb+ v3.4 Functionality Rolled In

Fast, Flexible, Low Memory end-of-day writes with TorQ 2.3

Chris McCracken data capture, kdb, kdb+, TorQ Leave a Comment

If you are given the option, is it to your advantage to switch on your initial choice? The Monty Hall problem: Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No 1, and the host, who knows what’s behind the …

Chris McCrackenFast, Flexible, Low Memory end-of-day writes with TorQ 2.3

TorQ 2.0 Released

Jonny Press data capture, kdb, kdb+, News, TorQ 1 Comment

We are proud to release version 2.0 of TorQ with some big functionality extensions.  The changes have stemmed from community feedback, customer installations and just what we thought would be nice to do.  You can download it here.  Enhancements include: new RDB (real time database) and WDB (writing database) processes.  These provide significant system architecture flexibility and can be used in combination …

Jonny PressTorQ 2.0 Released