TorQ kdb+ Framework

TorQ kdb+ Framework

TorQ is an open source framework that adds utilities and functionality on top of kdb+, allowing developers to focus on application business logic when building platforms

Find out more>>


TorQ Data Connectors

TorQ Data Connectors allow integration with commonly used marketdata providers and include message parsing, connection logic and schemas in kdb+

Find out more>>


Tech Connectors

TorQ Tech Connectors

TorQ Tech Connectors enable integration between kdb+ and common messaging, visualisation, monitoring, data stores and other third party systems

Find out more>>


TorQ Solutions

AquaQ TorQ Solutions

TorQ Solutions include a set of implementations built on top of kdb+ and TorQ incl TorQ Crypto, TorQ Betfair, TorQ IoT, TorQ Finance and TorQ CME

Find out more>>


Training

AquaQ TorQ Containers

TorQ Containers is a containerised solution built on top of kdb+ for a custom implementation either as a greenfield setup or using a customer’s existing kdb+ setup

Find out more>>


AquaQ TorQ Framework for kdb+       

TorQ is a framework that incorporates a suite of libraries and processes on top of kdb+. Together with kdb+, TorQ forms the basis of a production ready data capture, storage and analysis system. On average, TorQ will save the user 6 months of development work over starting a kdb+ project without TorQ. TorQ can be used with any sector that is dealing with vast quantities of real time data including IoT. TorQ integrates with Datadog (datadoghq.com) and Monit (mmonit.com) for monitoring and Grafana (grafana.com) for data visualisation. For user authentication TorQ is integrated with LDAP (ldap.com). TorQ can run with normal kdb+ licences or with kx on demand licences.


TorQ Data Quality System
TorQ DQS
, which includes pre-defined data quality checks and user-defined data quality breach alerts, allows developers to easily set up customizable checks on the quality of their kdb+ data. Examples include table schema consistency checks for streaming data and the detection of data attribute anomalies, both of which can be parameterized by the developer. The system consists of a Data Quality Checker component (DQC) and a Data Quality Engine component (DQE), with the results of the checks being stored to a results table. The results can be plugged into a client’s existing monitoring dashboards or can equally be plugged into a Grafana dashboard using AquaQ’s recently released Grafana Adaptor for kdb+. TorQ DQS can be used to monitor data quality in a TorQ-based kdb+ setup or separately in an independent kdb+ setup. Github >> here

TorQ PCAP Analyser
TorQ PCAP
Analyser incorporates packet sniffing into kdb+ to better identify problems over a network. The TorQ parser deconstructs PCAP files, which consist of sniffed packets in binary, and decodes the binary into a table in a kdb+ session. From there, the user can set up rules and alerts to track items of interest. TorQ PCAP Analyser can be run as part of a TorQ installation or it can be incorporated into a client’s existing kdb+ platform. Github >> here

TorQ was originally released as open source software in 2015 and the functionality has been added to since then with both internal AquaQ requirements and Client requirements. The framework is used for any new AquaQ Client engagements and any AquaQ product initiatives. It is in use in excess of 30 different clients, many of whom do not currently engage with AquaQ for consultancy work.

AquaQ Analytics also offers commercial support contracts for clients using our TorQ Framework.

Github Link: http://aquaqanalytics.github.io/TorQ/

AquaQ TorQ Data Connectors

AquaQ NASDAQ ITCH Connector

ITCH is the outbound protocol NASDAQ uses to communicate market data to its clients, that is, all information including market status, orders, trades, circuit breakers, etc. with nanosecond timestamps for each day and each exchange.

This connector consists of a kdb+/q parser to capture NASDAQ ITCH files into a kdb+ database.

Github Link: https://github.com/AquaQAnalytics/itch-parser

AquaQ TAQ Connector 

Daily TAQ (Trade And Quote) is a set of files that contain all trades and quotes for all issues listed and traded on US regulated exchanges for a single trading day. The Daily TAQ data is derived from the output of the CTA and UTP SIPs, tapes A, B, and C.

This connector is a feed Handler that loads TAQ (Trade and Quote) data files from the New York Stock Exchange.

Github Link: Currently being finalised for release.

AquaQ Activ Financial Connector

ACTIV is a global provider of real-time, multi-asset financial market data and solutions. ACTIV offers neutral, managed services for buy and sell-side firms and technology providers seeking a complete market data solution that encompasses both global content management and data delivery across the enterprise. With coverage spanning more than 150 equity and derivatives exchanges around the world ACTIV is a global provider of real-time, multi-asset financial market data and solutions.

This is a feed handler that captures Activ market data into a kdb+ database.

Github Link: private – please request details from info@aquaq.co.uk

AquaQ Nanex Connector

The Nanex framework NxCore is a high-performance ticker plant that brings the whole market to your workstation or desktop computer. NxCore excels in delivering and databasing all the quotes and trades transmitted by the exchanges, even in the hyper-active U.S. Option market (OPRA).

This feed handler captures Nanex market data into a kdb+ database.                 

Github Link: private – please request details from info@aquaq.co.uk

AquaQ TR MAPI Connector      

Thomson Reuters Matching API (MAPI) handlers provide ultra-low latency access to Thomson Reuters Matching market data and trading. They are implemented as easy-to-use yet flexible and powerful libraries that could be quickly integrated into client infrastructure.

Our MAPI TorQ Connector provides a wrapper to this framework to capture the data into a kdb+ database.        

Github Link: private – please request details from info@aquaq.co.uk

AquaQ IEX Connector

Investors Exchange (IEX) is a stock exchange based in the United States. It was founded in 2012 and launched as a national securities exchange in September 2016.

The AquaQ IEX Connector pulls data from the exchange into a kdb+ database.                

Github Link: private – please request details from info@aquaq.co.uk

AquaQ Tech Connectors

AquaQ q-REST

REST or RESTful API design (Representational State Transfer) is designed to take advantage of existing protocols. While REST can be used over nearly any protocol, it usually takes advantage of HTTP when used for Web APIs. This means that developers do not need to install libraries or additional software in order to take advantage of a REST API design. It is notable for its incredible layer of flexibility. has the ability to handle multiple types of calls, return different data formats and even change structurally with the correct implementation of hypermedia.

AquaQ q-REST is an open source REST service from AquaQ written in java 8 used to connect to an instance of kdb+ using JSON. The REST service can provide a single query to run or call a function predefined on the instance of kdb.

Github Link: https://github.com/AquaQAnalytics/q-REST

AquaQ Solace to kdb+ API       

Solace PubSub+ message brokers (solace.com) efficiently move information between all kinds of applications, users and devices, anywhere in the world, over all kinds of networks.

The AquaQ API allows kdb+ to communicate with any Solace brokers and provides a translation from most Solace datatypes to kdb+ and vice versa. Solace is becoming a real time messaging bus of choice for market data and other real time feeds (IoT and other industries also).             

Github Link: https://github.com/AquaQAnalytics/kdb-solace

AquaQ Grafana Adapter

Grafana (Grafana.com) is a multi-platform open source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources. It is expandable through a plug-in system. End users can create complex monitoring dashboards using interactive query builders.

The AquaQ Adapter allows users to visualise data from a kdb+ database in Grafana and sits on the official Grafana API Data Source page on the Grafana website.

Github Link: https://github.com/AquaQAnalytics/grafana-kdb-datasource-ws

AquaQ FIX Connector  

FIX (FInancial Information Exchange) is one of the most common protocols used within the finance industry. This AquaQ connector provides users with a shared library implementation of a FIX engine. The accompanying document also describes how to use this library to communicate with the FIXimulator tool for testing purposes.                

Github Link: https://github.com/AquaQAnalytics/kdb-fix-adaptor

AquaQ kafkaQ 

Apache Kafka is an open-source stream-processing software platform developed by LinkedIn and donated to the Apache Software Foundation, written in Scala and Java. It is a high-throughput, low-latency platform for handling real-time data feeds.

The AquaQ interface to kafka is a q language binding to kdb+ based on librdkafka.                     

Github Link: https://github.com/AquaQAnalytics/kafkaq

AquaQ MongoQ          

MongoDB is a cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with optional schemas.

This AquaQ Connector is a C shared lib along with associated q functions to allow push/pull access to mongoDB from within kdb+.

Github Link: https://github.com/AquaQAnalytics/mongoq

AquaQ Datadog to kdb+ API    

Datadog is a monitoring service for cloud-scale applications, providing monitoring of servers, databases, tools, and services, through a SaaS-based data analytics platform.

The AquaQ Datadog to kdb+ API hooks up Datadog to a kdb+ database for monitoring of kdb processes. This API is also part of the functionality available in the AquaQ TorQ Framework.    

Github Link: not available outside of TorQ Framework just yet.

AquaQ Monit to kdb+ API        

Monit is a small Open Source utility for managing and monitoring Unix systems. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations.

This Aqua API interfaces Monit to a kdb+ database for monitoring of kdb processes. This API is also part of the functionality available in the AquaQ TorQ Framework.     

Github Link: not available outside of TorQ Framework just yet.  

AquaQ IOT Connector  

The Internet of things(IoT) is a system of interrelated computing devices, mechanical and digital machines provided with unique identifiers and the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction. These devices output data in real time

A Feed Handler that takes in IoT data. Implemented in three client projects to date.       

Github Link: private – please request details from info@aquaq.co.uk

TorQ Solutions

AquaQ TorQ Finance Starter Pack        

This is a market data capture system that creates and uses random finance data. It consists of a real time random market data generator for Equities data and a TorQ set up for a kdb+ database.        

Github link: https://github.com/AquaQAnalytics/TorQ-Finance-Starter-Pack

AquaQ TorQ Betfair Starter Pack          

This is a sports exchange data capture system pulling real time sports exchange data from betfair.com. It consists of the AquaQ Betfair Connector and a TorQ set up for a kdb+ database. In addition to a system to capture betting exchange data, AquaQ previously completed a client project that involved an algorithm sending orders out to Betfair and tracking those orders.

Github link: https://github.com/AquaQAnalytics/TorQonnect-Betfair

AquaQ TorQ Crypto Starter Pack          

This is a market data capture system pulling real time crypto exchange data from a number of Crypto Currency Exchanges. It consists of the AquaQ Crypto Connector, a TorQ set up and a number of analytics for a kdb+ database.

Github Link: https://github.com/AquaQAnalytics/TorQ-Crypto

AquaQ TorQ Bikes IoT Starter Pack      

This is a data capture system pulling in city bicycles (london citi bikes, ny bikes, belfast bikes) IoT data. Consists of the AquaQ Bike Connector and a TorQ set up for a kdb+ database.

Github Link: https://github.com/AquaQAnalytics/TorQ-Bikes

AquaQ TorQ CME Starter Pack 

This is a market data capture system pulling in data from the CME (Chicago Mercantile Exchange). It consists of the AquaQ CME Connector and a TorQ set up for a kdb+ database.

Github Link: https://github.com/AquaQAnalytics/TorQ-CME

AquaQ IoT Starter Pack TorQ Solution  

This is a IoT data capture system pulling in real time data from IoT devices. It consists of the AquaQ  IoT Connector and a TorQ set up for a kdb+ database.

Github Link: private – please request details from info@aquaq.co.uk

TorQ kdb+ solutions on the Cloud        

AquaQ has implemented a number of TorQ Framework based solutions for clients on the cloud across finance and non-finance and have amassed experience across both AWS and Azure. Our Tradefeedr Joint Venture is fully implemented on Azure on the Cloud and from this we have extensive experience now of setting up a data platform on the cloud, making sure it is both secure and scalable. One of our Utilities client projects is also fully deployed on the cloud. Please request details from info@aquaq.co.uk

TorQ kdb+ Surveillance solutions         

AquaQ has implemented a number of TorQ Framework based Trader Surveillance solutions for finance clients and our teams are very experienced in this domain along with the associated case managements and investigations process. Please request details from info@aquaq.co.uk

Jamie GrantProducts – new