Description
DASH currently provides a set of patterns and distribution specifiers to control the distribution of tiles in a tiled matrix. However, there is currently no easy way to form blocks of local blocks, i.e., have super-blocks that are distributed with each super-block containing multiple local tiles. This can be beneficial for certain algorithms, e.g., QR factorization, to reduce the communication overhead.
Example (each entry is a tile, not a single value; the number denotes the unit owning it):
+---------------+
|0|0|1|1|0|0|1|1|
+---------------+
|0|0|1|1|0|0|1|1|
+---------------+
|2|2|3|3|2|2|3|3|
+---------------+
|2|2|3|3|2|2|3|3|
+---------------+
|0|0|1|1|0|0|1|1|
+---------------+
|0|0|1|1|0|0|1|1|
+---------------+
|2|2|3|3|2|2|3|3|
+---------------+
|2|2|3|3|2|2|3|3|
+---------------+
I am currently experimenting with 4-dimensional arrays, where the first two dimensions specify the tiles and the last two form the actual elements of the tiles. However, that comes at the cost of additional complexity as I lose the two-dimensional access in global memory, i.e., I cannot easily access any element at coordinates {x,y} without first figuring out the tile. We should have a way to specify such memory layouts natively in DASH.