Concatenation of result file from the partitions in // takes too long
Hi Boris
I've had the issue before but not sure I've mentioned it to you : the concatenation of the result files from the partitions on the cluster can take very long / too long.
I have an example (TELEMAC-3D) where the simulation had actually completed in just over 9hrs (as per the sortie file). I had specified a wall time of 36hrs and the run got killed before the 3D result file could be concatenated, let alone copied back to the run folder (which I originally thought was the main pitfall). It had not done the 2D result file either in this time.
In comparison, when I copied the partitioned files across to my laptop, it took gretel 9min to compile the 2D result file from the 48 partitions, and 18min for the 3D result file. Something does not seem right.
The input files for the simulation are too big to attach to this issue so I'll copy them tomorrow to /data/rd/projets/echanges.004/pour_Boris (think this gets erased every night). I've changed the duration of the simulation so it runs in about 11min. I've also changed the output frequency such that the number of time frames be similar to the original run.
In the meantime, I think I will use the --split, -x, --run options and do the merge manually...
Hope you can find somrthing. With many thanks - Noémie