Visualization of Cluster Log Data

The university runs a high-throughput cluster system which is part of the international science grid. The job scheduler used is CondorHT. There are many different computer pools at the university, each with changing size and load, depending on the job load and time of day.

The current visualization was a 2D grid of cells, each cell representing an active node in the pool. The color of the cell indicates the load (from low (red) to high (blue-green)) and some common jobs were identified by simple icons.

2D array of colored cubes
However, this data lacks the hierarchical structure of the pools and nodes and can be hard to read for novices.

Cluster Log Data

The log data of the cluster was very hierarchical. Available computers are grouped into pools, usually by the physical location on campus or assignment to a research lab. Each computer has a number of available nodes, usually one per CPU which can run jobs. Jobs can be split over multiple nodes and have attributes such as owner, timing, etc. Each node additionally has a 'load' which describes how well the CPU is used.


I wanted the visualization to resemble those 80s/90s cyberpunk ideas how a computer system looks like. I thought of those virtual neon-lit city blocks -- this fit well with the hierarchical structures of the cluster. Each computer became a skyscraper, the number of nodes describing its height. Powerful computers are tall skyscrapers, more standard ones smaller buildings. Next, these computers are grouped by their pools -- neighbourhoods began to develop with high-rises in some small neighbourhoods and more pedestrian lower but large suburban neighbourhoords.