For many of us, this month is about beginning the first delivery cycle of the year knowing that we have a lot of work to get through before the December change freeze hits us again. Requirements year on year usually include increasing data volume, throughput, availability and access while not wanting to impact on performance, availability and access. Often kdb+ …
Level 2 Storage Formats
Storing an order book can be tricky due to the large number of orders and prices that continually change throughout the day. Here we investigate and compare three common methods of storing an order book, assessing their benefits and limitations. To do that, the TorQ-CME pack was used to process CME MDP 3.0 FIX data on Soybean Futures for 10 …
AquaQuarantine kdb+ Tech Talks Round 2
We gave some talks a few months ago which seem to have gone down well. We’ve decided to go again with a short series of 3, including a guest talk from Nick Psaris on his new book Fun Q. If you’d like to sign up for any of these sessions, please use the form below. If you have trouble accessing …
AquaQuarantine kdb+ Tech Talks
These are some strange times. We hope everyone and their families are safe and well. On a positive note, it does provide a good opportunity to run some technical training sessions, which we’ve been doing internally for the last few weeks. We’ve decided to open these up publicly as everyone else is in the same “working from home” boat. We’ll …
Monitoring a TorQ System With Datadog
Datadog is a monitoring service for cloud-scale applications, providing monitoring of servers, databases, tools, and services, through a SaaS-based data analytics platform. This post illustrates how we can create a dashboard, monitors and alerts to track metrics and events from a TorQ Data Capture System.
Scaling kdb+ with TorQ and Google Cloud
Instead of having a static infrastructure that is unresponsive to your changing usage, Google’s cloud platform enables the infrastructure to dynamically scale along with your usage. This means you only pay for what you are using, when you are using it. We’ll create a simple proof of concept demonstrating this scaling behaviour when used with kdb+. This will utilize some …
Kafka with kdb+ and Java
Kafka is a distributed messaging service used to achieve a low-latency, high-throughput transfer of data using persistent logs to prevent loss of data. This blog will: Illustrate the various components used in Kafka. Show how Kafka can be used to share data between different technologies, for example, a Java producer and a kdb+ consumer. Highlight a use case for Kafka …
Grafana kdb+ Adaptor Update
One of the most requested features to our kdb+ Grafana adaptor was the ability to create panels from the results of server side functions. This most recent update to the adaptor continues to build on the SimpleJSON datasource created by Grafana by working around the limitation of the single drop down menu for queries, whilst also allowing parameters to be supplied. The adaptor is also integrated into TorQ …
Grafana – kdb+ adaptor
AquaQ Analytics are often involved in projects using different visualisation tools connected to kdb+. We have recently spent some time reviewing and then implementing a simple Grafana-kdb+ adaptor. This open-source script has been developed using the inbuilt Grafana SimpleJSON datasource and comes with the capability of table and time series visualisation and manipulation. Instructions for installation and usage can be …
AquaQ releases q-REST Web Interface
AquaQ Analytics is pleased to announce the release of q-REST v1.0 – a RESTful web interface for kdb+. This open source package has been designed as a convenient way for web clients to make requests to a kdb+ instance using the JSON format. The package can be easily integrated with Swagger UI, Docker and standard authentication methods such as SSO, allowing users to easily implement the …