In general, +1, don't see why not.
Q -- is it something that you have encountered while doing algebra? I.e.,
do you need the sorted DRM to continue algebraic operations between
optimizer barriers, or you just need an RDD as the outcome of all this?
if it is just an RDD, then you could just do a spark-supported sort, that's
why we have an drm.rdd barrier (spark-specific). Barrier out to spark RDD
and then continue doing whatever spark already supports.
Another potential issue is that matrices do not generally imply ordering or
formation of intermediate products, i.e., we inside optimizer, you might
build a pipeline that implies ordered RDD in Spark sense, but there is no
algebraic operator consuming sorted rdds, and no operator that guarantees
preserving it (even if it just a checkpoint). This may create ambiguities
as more rewriting rules are added. This is not a major concern.
On Tue, Sep 5, 2017 at 2:24 PM, Trevor Grant <[EMAIL PROTECTED]>