← Back to all posts

PipelineDB 0.8.6


PipelineDB 0.8.6 is here, download it now!

In addition to various bug fixes and performance improvements, here are the highlights of this release:

Removed GIS From Core Codebase

We removed the PostGIS-compatible module that previously was built and installed by default with the PipelineDB core. While this module is still fully compatible with PipelineDB, most of our users were not using GIS so we removed it because it has dependencies. Nearly all PipelineDB installation issues were caused by these dependencies, so building and installing PipelineDB is much simpler now.

User-Specified Primary Keys on Continuous Views

In release 0.8.5, we added primary keys to continuous views' materialization tables, mainly for use within PipelineDB's internals. Users requested that they be able to specify one of the continuous view's existing columns to serve as the primary key, rather than having an additional column implicitly added. 0.8.6 adds support for this, which is accessible via the pk storage parameter used when creating a continuous view:

-- Implicitly creates an 8-byte integer primary key column
CREATE CONTINUOUS VIEW v0 AS
    SELECT x, COUNT(*) FROM stream GROUP BY x;

-- x will be used as the primary key and no additional columns will be created
CREATE CONTINUOUS VIEW v1 WITH (pk='x') AS
    SELECT x, COUNT(*) FROM stream GROUP BY x;

Support for Creating Indices Directly on Continuous View

Because continuous views are physically backed by materialization tables (named <continuous view name>_mrel, previously any indices had to be created on this table. This had two complications. First, it isn't necessarily obvious that this table exists or what it's called, as it's mainly for internal use. Secondly, certain continuous views actually store intermediate values in the materialization table that are finalized at read time. For this case it wasn't even possible to index such columns without using some kind of sorcery.

0.8.6 supports creating indices directly on continuous views, and knows how to properly index columns that store unfinalized values by using expression indices.

Tab-Completion in pipeline Client for PipelineDB-Specific Commands

The pipeline client now supports tab completion for PipelineDB-specific commands such as CREATE CONTINUOUS VIEW, CREATE STREAM, ACTIVATE, EXPLAIN CONTINUOUS VIEW, etc. Hey, the little things matter :)

Breaking Changes

We've made some optimizations to continuous views' internal indexing behavior, so this release is not compatible with previous versions. To migrate your continuous views to a 0.8.6 deployment, there are two options. The most straightforward is to use pipeline-dump/pipeline-restore.

For the more adventerous, you can REINDEX all of your materialization tables (<continuous view name>_mrel) before writing to any streams. After installing PipelineDB 0.8.6, for each of your continuous views, run:

REINDEX TABLE <continuous view name>_mrel;

Thanks!

We'd like to give our thanks to open-source contributors cleaton, yannick, guanquan, and myechuri for their contributions to this release. We're humbled by our users taking the time to make PipelineDB better for everyone. Thank you!

What's Coming Next

Thanks to the engagement we have from our community of users, and the invaluable feedback and suggestions they provide us with, PipelineDB has matured significantly since its initial release. It is becoming quite stable, and our next slated release is 0.9.0. The biggest improvement that will ship with 0.9.0 is full compatibility with PostgreSQL 9.5. We expect to ship it relatively soon, so stay tuned!