Posts Tagged ‘KDB+’

Cheap air quality monitoring

January 16, 2017

About a year ago I set up an air quality monitoring station based on the Air Quality Egg product by Wicked Device (see also my previous post on AQE). It was working during the winter season, providing real time data on PM2.5 concentrations near my house. This graph shows some of the data collected.

This worked OK, except that the data from the device were uploaded to Xively and available only from there. Xively was providing this service free as it was fulfilling an old commitment from one of their acquisitions. It was not high on their priority list and the service was frequently down. I decided to build my own device then to have full control over the process – from collecting the data to displaying them on a public web page. And to have some creative fun. The result works well (at least at the time of the writing) and cost less than $80 in materials, including a Raspberry Pi that I used. I think building a PM2.5 monitoring station in a way similar to what I describe below would make an excellent high school project.

(more…)

KDB+, Elm and web sockets

July 15, 2014

In the KDB+ implementation of the Conway’s Game of Life that I presented in my previous post there was one element missing – a GUI that would display the results of the simulation. Since I have been planning to have a look at Elm for a while, I checked if I can set up KDB+ talking to an Elm application and it turned out it is very easy to do – with web sockets.

Elm is a functional programming language that compiles to HTML, JavaScript and CSS code intended to run in a browser. The declared paradigm for Elm is Functional Reactive Programming. FRP is not exactly what it used to be back in the days of Conal Elliot and Paul Hudak’s original work. It is now more of a buzzword covering more and more semantic area. The key concept in the  Elm’s take on FRP are signals. A Signal a type in Elm represents a value of type a that changes in time. Signals can be thought of as streams of discrete events carrying values. They can be combined, filtered, counted and so on. Ultimately we obtain a value of type Signal Element that can be displayed in the browser (possibly in a div if one embeds an Elm application this way). The Elm site contains a very convincing argument on why all this is a good idea.

(more…)

Game of Life with Enterprise Components

May 20, 2014

DEVnet (the company I work for) has decided to release one of its products – Enterprise Components (called EC below) – as free software. Since I contributed some code to it I would like to advertise it a bit on this blog.

What are Enterprise Components?

EC is a collection of libraries and tools that support building systems based on KDB+ – a database made by Kx Systems. Traditionally KDB+ has been mostly used for time series analytics in finance industry. Part of the reason for that was that the commercial license for KDB+ was very expensive. However, some two months ago Kx announced that they now allow commercial use of the free-of-cost 32-bit version of their product. This opened a path for KDB+ applications in areas well beyond institutions with deep pockets.

The main limitation of the 32-bit version is the amount of data it can handle – about 1GB. This limitation is per-process though and since KDB+ has a built-in inter-process communication protocol one can create very capable systems using the free version just running as many instances as needed.

Enterprise Components can be thought of as a layer above KDB+. EC provide logging, process management, configuration management, user access control, system monitoring and more – all that is needed if you want to build a larger scale system. Typical use cases are covered in the standard components provided with EC so that one can define a basic system with data feeds, in-memory database backed by on-disk journals and storage and archiving at the end of day with configuration only.

(more…)

Q – six months later

February 13, 2010

It has been six months since I started to earn my living by writing code in Q. I am still not a guru (a Q god as they call it in Q circles), but not a complete newbie either. (more…)