xnn.common.distributed.all_reduce_sum

xnn.common.distributed.all_reduce_sum(values, device)

Sum a list of scalars over all ranks (identity outside a distributed run).

Parameters:

• values (list of float) – Per-rank partial sums, in the same order on every rank.

• device (torch.device) – This rank’s compute device, which selects where the collective runs.

Returns:

The global sums, identical on every rank.

Return type:

list of float