Methods for storing text on-disk in kdb+

Calum Mackenzie kdb+

q has both the symbol and character atomic datatypes. Both symbols and lists of characters (commonly referred to as strings) are used for working with text data. The two appear extensively throughout q code but in any given situation there are reasons for strongly favouring one over the other. This post will start by covering those reasons and then go …

Calum MackenzieMethods for storing text on-disk in kdb+

TorQ Logging with Google Cloud Platform

Eoin Teague cloud, kdb+, TorQ Leave a Comment

TorQ provides detailed and thorough standard output, error and usage logging for multiple processes. These comprehensive logs are essential to system development and monitoring. However more detailed logs can increase the time and effort required to examine log files. Logs aggregation can streamline log analysis, by consolidating log data from multiple components or applications into a centralised location. Log aggregation …

Eoin TeagueTorQ Logging with Google Cloud Platform

Is your kdb+ data a mesh?

Gary Davies data, kdb+

No it’s not a typo, this isn’t a blog about how to keep data clean, rather this will explore the concept of a data mesh and more specifically our thoughts on it, in relation to kdb+. In the world of data we are often presented with new trends in the storage, distribution and access of data. Previous iterations include data …

Gary DaviesIs your kdb+ data a mesh?

AquaQ Grafana 8.0 plugin for kdb+

Paul Teague kdb+

AquaQ is pleased to preview an upcoming release of a new product. AquaQ have developed a new Grafana 8.0 kdb+ backend plugin. This plugin has been developed using https://github.com/grafana/grafana-plugin-sdk-go.  The plugin offers the capability for table and time series visualisation of queried data along with support for Grafana variables, enhanced TLS security, query capability, native support for Grafana’s alerting framework …

Paul TeagueAquaQ Grafana 8.0 plugin for kdb+

All aboard the AquaQ ARK (Architecture Review for kdb+)

Gary Davies cloud, kdb, kdb+

For many of us, this month is about beginning the first delivery cycle of the year knowing that we have a lot of work to get through before the December change freeze hits us again. Requirements year on year usually include increasing data volume, throughput, availability and access while not wanting to impact on performance, availability and access. Often kdb+ …

Gary DaviesAll aboard the AquaQ ARK (Architecture Review for kdb+)

TorQ Roadmap 2022

Jonny Press cloud, kdb+, TorQ

We are currently assisting a number of clients with the move towards cloud, and cloud capabilities feature heavily in the development pipeline for TorQ, our kdb+ production framework. The move to cloud can be a “lift-and-shift” approach, or it can be an opportunity to re-engineer the solution. The case for re-engineering parts or all of an incumbent solution are strong. …

Jonny PressTorQ Roadmap 2022

CI/CD & TorQ-Pipeline

Liam O'Connor kdb+, TorQ

Many application development lifecycles have moved to more modern cycle approaching full automation. The purpose of this blog is to demonstrate TorQ-Pipeline, an example using TorQ of a modern kdb+ CI/CD (Continuous Integration/Continuous Delivery/Deployment) pipeline. Before DevOps culture, the software cycle was heavily siloed. The goal of CI/CD is to break down the barriers between the different teams involved in …

Liam O'ConnorCI/CD & TorQ-Pipeline

MATLAB, kdb+ and Streaming Data

Liam O'Connor kdb+

Introduction Recent projects at AquaQ have required the ability to receive streaming data to MATLAB from a vanilla kdb+ tickerplant. The aim therefore was to create a simple method for this with the ability to work over as many platforms as possible. The examples included in this blog use the R2021a version of MATLAB and kdb+ 4.0. Java API for kdb+  …

Liam O'ConnorMATLAB, kdb+ and Streaming Data

Our database is currently busy, please hold…

Gary Davies kdb+

In a data driven world, the ability to access and retrieve data efficiently is critical. This means data needs to be readily available to users, and in the kdb+ world this can present a challenge. kdb+ being somewhat single threaded can result in heavy users acting greedily on data processes. In AquaQ we are hearing more from our clients about …

Gary DaviesOur database is currently busy, please hold…

Automating Data Egress from kdb+ into Google BigQuery

Matthew Clark data capture, kdb+, TorQ Leave a Comment

kdb+ tick data capture software has been largely popular in banks, hedge funds and other financial institutions for years. Its kdb+ framework permits incredibly fast querying of data largely due to its vectorised querying language and efficient storage methods. A large cost component in any kdb+ implementation is always storage costs. Storage costs have been a big driving factor towards …

Matthew ClarkAutomating Data Egress from kdb+ into Google BigQuery