AquaQuarantine kdb+ Tech Talks

Jonny Press datablog, kdb, kdb+, TorQ, training 5 Comments

These are some strange times. We hope everyone and their families are safe and well. On a positive note, it does provide a good opportunity to run some technical training sessions, which we’ve been doing internally for the last few weeks. We’ve decided to open these up publicly as everyone else is in the same “working from home” boat. We’ll …

Jonny PressAquaQuarantine kdb+ Tech Talks

Monitoring a TorQ System With Datadog

Ellen Mooney Datadog, kdb, kdb+, TorQ Leave a Comment

Datadog is a monitoring service for cloud-scale applications, providing monitoring of servers, databases, tools, and services, through a SaaS-based data analytics platform. This post illustrates how we can create a dashboard, monitors and alerts to track metrics and events from a TorQ Data Capture System.

Ellen MooneyMonitoring a TorQ System With Datadog

Scaling kdb+ with TorQ and Google Cloud

Glen Smith kdb, kdb+, TorQ Leave a Comment

Instead of having a static infrastructure that is unresponsive to your changing usage, Google’s cloud platform enables the infrastructure to dynamically scale along with your usage. This means you only pay for what you are using, when you are using it. We’ll create a simple proof of concept demonstrating this scaling behaviour when used with kdb+. This will utilize some …

Glen SmithScaling kdb+ with TorQ and Google Cloud

Kafka with kdb+ and Java

Paul Doherty kdb, kdb+ 3 Comments

Kafka is a distributed messaging service used to achieve a low-latency, high-throughput transfer of data using persistent logs to prevent loss of data. This blog will: Illustrate the various components used in Kafka. Show how Kafka can be used to share data between different technologies, for example, a Java producer and a kdb+ consumer. Highlight a use case for Kafka …

Paul DohertyKafka with kdb+ and Java

Grafana kdb+ Adaptor Update

Dylan Creighton kdb, kdb+ 1 Comment

One of the most requested features to our kdb+ Grafana adaptor was the ability to create panels from the results of server side functions. This most recent update to the adaptor continues to build on the SimpleJSON datasource created by Grafana by working around the limitation of the single drop down menu for queries, whilst also allowing parameters to be supplied. The adaptor is also integrated into TorQ …

Dylan CreightonGrafana kdb+ Adaptor Update

Grafana – kdb+ adaptor

AquaQ Admin kdb, kdb+ 3 Comments

AquaQ Analytics are often involved in projects using different visualisation tools connected to kdb+.  We have recently spent some time reviewing and then implementing a simple Grafana-kdb+ adaptor. This open-source script has been developed using the inbuilt Grafana SimpleJSON datasource and comes with the capability of table and time series visualisation and manipulation. Instructions for installation and usage can be …

AquaQ AdminGrafana – kdb+ adaptor

AquaQ releases q-REST Web Interface

Michael McParland kdb, kdb+, TorQ Leave a Comment

AquaQ Analytics is pleased to announce the release of q-REST v1.0 – a RESTful web interface for kdb+. This open source package has been designed as a convenient way for web clients to make requests to a kdb+ instance using the JSON format. The package can be easily integrated with Swagger UI, Docker and standard authentication methods such as SSO, allowing users to easily implement the …

Michael McParlandAquaQ releases q-REST Web Interface

TorQ Release v3.3 – kdb+ 3.6

AquaQ Admin kdb, kdb+, TorQ Leave a Comment

We are delighted to announce release v3.3 of TorQ, the latest installment of our kdb+ framework; which can be found here. The TorQ v3.3 release has upgraded to run with kdb+ 3.6, whilst maintaining backward compatibility with prior versions.  The main changes are outlined below. Concurrent Synchronous Requests As part of this update, the gateway in TorQ supports the deferred response …

AquaQ AdminTorQ Release v3.3 – kdb+ 3.6

As You Bike It: Mapping Belfast Bike Usage

Ryan McCarron data capture, datablog, kdb, kdb+, TorQ Leave a Comment

We recently noted the ability to use our TorQ package to collect data on our local Belfast Bikes transport network. So how does our cycle network look after collecting for a while? We recorded data on the status of each station in the network for 6 months, and we can now use this data to observe how the population of …

Ryan McCarronAs You Bike It: Mapping Belfast Bike Usage

kdb+ Feedhandler Development

AquaQ Admin data capture, datablog, kdb, kdb+ Leave a Comment

Setting up a kdb+ database is relatively straight forward.  The TorQ Starter Pack is a good place to start.  Two of the trickier bits are: Estimating how much memory your dataset will require Writing applications (generally called “feedhandlers”) to insert data into the database For (1) we’ve written previously on some database set up utilities used to estimate the size of a dataset …

AquaQ Adminkdb+ Feedhandler Development