Authors: Eric Sol and Ferry Tap.
Slides: [1], [2]

Computational modelling is the art & science of computer based simulations to better understand complex objects such as a brain, a molecule or a combustion engine. The majority of these simulations
demand huge computing infrastructures yet still take days or even weeks to complete.

Dacolt is a small, hightech firm based in Maastricht that offers software and services for CFD (Computational Fluid Dynamics) modelling of industrial combustion applications. Well known customers
include Renault, Volkswagen and Air Liquide. Dacolt is lead by Dr. Ferry Tap (1974), a graduate from Delft Technical University.

Solide is a small, innovative web development company based in Echt. Solide is lead by Eric Sol (1961), an early adopter of Internet and former ICT-manager in Dutch Higher Education.

In the past years both companies have developed a web based platform that enables customers to manage these complex simulations from the webbrowser, e.g. without knowledge of the underlying infrastructure.
Innovative as it is, our next goal was to gain flexibility from the huge numeric output files, in the order of gigabytes, that are generated by these simulations.

Databases are designed for manipulating data, so we investigated the possible use of (free available) no-SQL and SQL-databases. We selected PostgreSQL as best overall product. Since it is virtually
impossible to select attractive cross-sections from numerical data we also needed a tool to visualize the data. We selected Flotr, a javascript plot library, for that.

Our resulting application has facilities for importing, structuring and visualizing large amounts of data. Current databases are in the order of gigabytes but cross-sections can be plotted in the
web browser in less then one second. We investigated different alternatives to handle such large databases (default partitioning, custom partitioning) but always ended up fighting limitations like number of table locks or number of columns. Our final solution is based on the storage of arrays in tables with fixed positions for every parameter in the dataset. The positions are stored in a lookup-like table with all parameters within the dataset. This gave us both flexibility and speed. The Flotr library, although being the best starting point for visualisation, needed attention too. We fixed our share of bugs and spent quite a few hours on drawing logarithmic scales.

Yes, we will show pictures!

Remark: the resulting application is not open source since it is a major business driver for Dacolt. However, the modifications necessary to make Flotr draw logarithmic graphs correctly have all been contributed back to the project, along with many critical bugfixes and other improvements.