The Road So Far (TorQ RM 2018)

Stephen McDonald data capture, kdb+, TorQ Leave a Comment

With 2018 well and truly underway, our development team has set off on a journey along the 2018 TorQ Roadmap. We previously posted a compilation of requested features and desired improvements for our TorQ framework, which has provided us with a range of mini-projects to take on. We are keen on receiving prioritisation requests, if a feature on the roadmap …

Stephen McDonaldThe Road So Far (TorQ RM 2018)

As You Bike It: Mapping Belfast Bike Usage

Ryan McCarron data capture, datablog, kdb, kdb+, TorQ Leave a Comment

We recently noted the ability to use our TorQ package to collect data on our local Belfast Bikes transport network. So how does our cycle network look after collecting for a while? We recorded data on the status of each station in the network for 6 months, and we can now use this data to observe how the population of …

Ryan McCarronAs You Bike It: Mapping Belfast Bike Usage

TorQ 2018 Roadmap

Jonny Press data capture, kdb+, TorQ Leave a Comment

Below is our current TorQ roadmap for 2018. We welcome any additional suggestions, or prioritisation requests. Data Manipulation Library The point here is to produce a set of utilities useful for analysing and manipulating datasets. The target audience is new kdb+ developers or business users. Visualisation products (e.g. Panopticon) are probably good target applications as it is useful to pivot, …

Jonny PressTorQ 2018 Roadmap

Smart Belfast Collaborative Challenge & Belfast Bikes

AquaQ Admin data capture, kdb+, News Leave a Comment

With the recent launch of Smart Belfast Collaborative Challenge by Invest NI and Belfast City Council we thought we’d write a short blog about the existing transport success story of BelfastBikes. If any companies are interested in talking to AquaQ Analytics about their proposed collaborative network and assistance on data please contact us on info@aquaq.co.uk We noticed an API end-point …

AquaQ AdminSmart Belfast Collaborative Challenge & Belfast Bikes

kdb+ Feedhandler Development

AquaQ Admin data capture, datablog, kdb, kdb+ Leave a Comment

Setting up a kdb+ database is relatively straight forward.  The TorQ Starter Pack is a good place to start.  Two of the trickier bits are: Estimating how much memory your dataset will require Writing applications (generally called “feedhandlers”) to insert data into the database For (1) we’ve written previously on some database set up utilities used to estimate the size of a dataset …

AquaQ Adminkdb+ Feedhandler Development

TorQ Release v3.1 – Kafka, Data Replay and Subscriber Cutoff

Jamie Grant data capture, kdb, kdb+, TorQ 1 Comment

We are delighted to announce release v3.1 of TorQ, the latest instalment of our kdb+ framework. The first of our great new additions is kafka.q, which provides q language bindings for the ‘distributed streaming platform’ Apache Kafka, a real time messaging system with persistent storage in message logs. An application architecture built around Kafka could dispense with a tickerplant component, …

Jamie GrantTorQ Release v3.1 – Kafka, Data Replay and Subscriber Cutoff

Data Replay and Backtest in TorQ

Allan Moore data capture, kdb+, TorQ Leave a Comment

TorQ has a new utility for replaying historical data into real-time data processes, datareplay.q.  Replaying data is usually the first step towards allowing you to backtest.  datareplay.q builds a table of upd function calls like those generated by a tickerplant, but instead, using a historical database as the data source, making it simple to test new or existing real-time subscribers …

Allan MooreData Replay and Backtest in TorQ

Cloud kdb+

Jonathon McMurray data capture, datablog, kdb, kdb+ Leave a Comment

When setting up a kdb+ production system, you may not always have access to the hardware you need. In such instances, it can be helpful to turn to “the cloud”. There are a number of potential benefits to using a cloud system instead of a local server. For example, important server-related duties such as security, failover and data redundancy will …

Jonathon McMurrayCloud kdb+

New TorQ-FX Released

Jamie Grant data capture, kdb, kdb+, TorQ Leave a Comment

For anyone who has used TorQ, you will most likely be familiar with the Finance Starter Pack. The starter pack is an example data capture system that is based on randomly generated equity data. It was created with the purpose of showing how to set up an example TorQ installation and how different applications can be built and deployed on …

Jamie GrantNew TorQ-FX Released

kdb+ Database Setup Utilities

AquaQ Admin data capture, datablog, kdb, kdb+, TorQ 1 Comment

This blog post describes tools to help setup a new kdb+ database. The tools allow you to: calculate the expected memory requirements of the database check columns have been typed correctly in order to avoid sym file bloat Background The volume of data that a kdb+ process can store in memory is finite and must be estimated and considered when designing …

AquaQ Adminkdb+ Database Setup Utilities