The Kx/Thalesians/WBS course on big data, high-frequency data, and machine learning with kdb+/q

Jan Novotny explaining the vectorised approach to tree-based Machine Learning problems at the 14th Quantitative Finance Conference in Nice

Overview

q is a programming language for array processing, developed by Arthur Whitney on the basis of Kenneth E. Iverson’s APL. The kdb+ database built on top of q is a de facto standard technology for dealing with rapidly arriving, high-frequency, big data.

kdb+/q has taken the world of electronic, including algorithmic, trading by storm. It is used by numerous sell-side and buy-side institutions, including some of the most successful hedge funds and electronic market makers.

Beyong the world of electronic trading, kdb+/q is used in retail, gaming, manufacturing, telco, IoT, life sciences, utilities, and aerospace industries.

Your course will take you through the foundations of kdb+/q and explain why it is a language of choice for Big Data, high-frequency data, and real-time event processing.

We shall explain how to work with tables and q-sql effectively, how to set up tickerplants, real-time, and historical instances, and how to apply kdb+/q to machine learning problems.

We shall consider advanced applications to tree-based regression and classification, random forests, deep learning, Google DeepMind and Monte Carlo search, producing demonstrations on real-life data examples.

Instructors

  • Dr. Paul Bilokon
  • Dr. Jan Novotny

Venue

Level39 of One Canada Square, Canary Wharf, London’s premier fintech hub.

Sponsors

Since the company’s inception, Kx‘s singular goal has been to provide its customers with the fastest, most efficient, and most flexible tools for processing real-time and historical data. This focus has enabled us to become the worldwide leader of in-memory time-series databases.

WBS Training Ltd (World Business Strategies) organises workshops and conferences for the capital markets and treasury divisions of investment companies worldwide, with all our efforts centered solely on the education of our clients. WBS Training does not operate to present dozens of events every year. Instead we select only the most innovative, pertinent and dynamic subjects, thus bridging the gap between the latest theoretical developments through to proven practical trading floor requirements. Therefore, we aim to ensure that such requirements can be effectively implemented in the real financial world.

Registration

Schedule

TimeDay 1Day 2
08:30 – 09:00Registration and welcome, a tour of Level39Registration and welcome
09:00 – 10:00Lecture 1: Foundations of kdb+ and the q programming languageLecture 1: Data science and machine learning crash course
10:00 – 10:30Tutorial 1Tutorial 1
10:30 – 11:00Coffee breakCoffee break
11:00 – 12:00Lecture 2: Working with tables and q-sqlLecture 2: Tree-based regression and classification, random forests
12:00 – 12:30Tutorial 2Tutorial 2
12:30 – 13:30LunchLunch
13:30 – 14:30Lecture 3: Big data in kdb+/qLecture 3: Neural networks in kdb+/q
14:30 – 15:00Tutorial 3Tutorial 3
15:00 – 15:30Coffee breakCoffee break
15:30 – 16:30Lecture 4: Tickerplant architecture for data capturesLecture 4: Applications of machine learning in kdb+/q; Google DeepMind and Monte Carlo search
16:30 – 17:00Tutorial 4Tutorial 4
17:00 – 18:00LabLab

Syllabus

  • Foundations of kdb+ and the q programming language
  • Working with tables and q-sql
  • Big data in kdb+/q
  • Tickerplant architecture for data captures
  • Data science and machine learning crash course
  • Tree-based regression and classification, random forests
  • Neural networks in kdb+/q
  • Applications of machine learning in kdb+/q; Google DeepMind and Monte Carlo search

What to bring

Please bring your own laptop with a development version of 32-bit Personal Edition of kdb+/q version 3.6 installed. You can download it from https://kx.com/connect-with-us/download/

Bibliography

Your course is designed to be self-contained. However, if you would like to read up on the content before, during, and/or after the course, we recommend the following books:

  • Jeffry A. Borror. q For Mortals Version 3: An Introduction to q Programming. q4m LLC, 2015.
  • Nick Psaris. Q Tips: Fast, Scalable and Maintainable Kdb+. Vector Sigma, 2015.
  • Paul Bilokon, Jan Novotny, Aris Galiotos, Frederic Deleze. Machine Learning and Big Data with kdb+/q. Wiley, 2019.