kdb+ Feedhandler Development

Stephanie Woodings data capture, datablog, kdb, kdb+ Leave a Comment

Setting up a kdb+ database is relatively straight forward.  The TorQ Starter Pack is a good place to start.  Two of the trickier bits are: Estimating how much memory your dataset will require Writing applications (generally called “feedhandlers”) to insert data into the database For (1) we’ve written previously on some database set up utilities used to estimate the size of a dataset …

Stephanie Woodingskdb+ Feedhandler Development

TorQ Release v3.1 – Kafka, Data Replay and Subscriber Cutoff

Jamie Grant data capture, kdb, kdb+, TorQ Leave a Comment

We are delighted to announce release v3.1 of TorQ, the latest instalment of our kdb+ framework. The first of our great new additions is kafka.q, which provides q language bindings for the ‘distributed streaming platform’ Apache Kafka, a real time messaging system with persistent storage in message logs. An application architecture built around Kafka could dispense with a tickerplant component, …

Jamie GrantTorQ Release v3.1 – Kafka, Data Replay and Subscriber Cutoff

Data Replay and Backtest in TorQ

Allan Moore data capture, kdb+, TorQ Leave a Comment

TorQ has a new utility for replaying historical data into real-time data processes, datareplay.q.  Replaying data is usually the first step towards allowing you to backtest.  datareplay.q builds a table of upd function calls like those generated by a tickerplant, but instead, using a historical database as the data source, making it simple to test new or existing real-time subscribers …

Allan MooreData Replay and Backtest in TorQ

Cloud kdb+

Jonathon McMurray data capture, datablog, kdb, kdb+ Leave a Comment

When setting up a kdb+ production system, you may not always have access to the hardware you need. In such instances, it can be helpful to turn to “the cloud”. There are a number of potential benefits to using a cloud system instead of a local server. For example, important server-related duties such as security, failover and data redundancy will …

Jonathon McMurrayCloud kdb+

New TorQ-FX Released

Jamie Grant data capture, kdb, kdb+, TorQ Leave a Comment

For anyone who has used TorQ, you will most likely be familiar with the Finance Starter Pack. The starter pack is an example data capture system that is based on randomly generated equity data. It was created with the purpose of showing how to set up an example TorQ installation and how different applications can be built and deployed on …

Jamie GrantNew TorQ-FX Released

kdb+ Database Setup Utilities

Aidan O'Gorman data capture, datablog, kdb, kdb+, TorQ 1 Comment

This blog post describes tools to help setup a new kdb+ database. The tools allow you to: calculate the expected memory requirements of the database check columns have been typed correctly in order to avoid sym file bloat Background The volume of data that a kdb+ process can store in memory is finite and must be estimated and considered when designing …

Aidan O'Gormankdb+ Database Setup Utilities

kdb+ Online Training Updated- Architecture

Jonny Press data capture, kdb, kdb+, TorQ, training Leave a Comment

We recorded our recent kdb+ Architecture Workshop in London, and we’ve added the video and slide deck as a bonus module on our kdb+ Bootcamp Online Training Course.  Above is a snapshot – a three hour in-depth workshop in 15 seconds.  Topics covered include: the basics of data capture extending kdb+ tick and alternative strategies scaling throughput versus latency removing …

Jonny Presskdb+ Online Training Updated- Architecture

Faster Recovery with kdb+ tick

Jonathon McMurray data capture, kdb, kdb+, TorQ Leave a Comment

The standard tickerplant (tick.q) from kdb+tick receives updates from feeds and pushes these updates to subscribing processes, as well as writing them to the log file. The tickerplant (TP) will count the number of messages received from the feed and written to the log file (in a variable .u.j) as well as the number of messages pushed to subscribers (in …

Jonathon McMurrayFaster Recovery with kdb+ tick

Free of Charge kdb+ Architecture Workshop NYC

AquaQ Admin data capture, datablog, kdb, kdb+, News, TorQ, training Leave a Comment

We were discussing our launch party in NYC and were kicking around ideas of what to do before it.  Some of us wanted to go to the Metropolitan Museum of Art; some of us wanted to go for a long lunch and maybe a stroll through Central Park; some of us wanted to run a kdb+ Architecture Workshop. So here we are. …

AquaQ AdminFree of Charge kdb+ Architecture Workshop NYC

TorQ User Group

AquaQ Admin data capture, datablog, kdb, kdb+, TorQ Leave a Comment

We have recently created a TorQ user group, for questions and discussions on TorQ (our kdb+ framework).  We will also be using it for more general data capture architecture considerations, and we’ll be publishing coding challenges every two weeks. We hope it will become a useful resource!

AquaQ AdminTorQ User Group