如何使用Stream Parallel Java进行矩阵计算？

Question

我正在尝试使用多维数组（[verybigrow][2]）创建矩阵算术运算方法。我是新手，我只是找不到我做错了什么。如果有人帮助我告诉我它是什么，我真的很感激。

    try {

        Stream<String> Matrix = Files.lines(Paths.get(file)).parallel();

        String[][] DataSet = Matrix.map(mapping -> mapping.split(",")).toArray(String[][]::new);

        Double[][] distanceTable = new Double[DataSet.length - 1][];

        /* START WANT TO REPLACE THIS MATRIX CALCULATION WITH PARALLEL STREAM RATHER THAN USE TRADITIONAL ARRAY ARITHMETICS START  */

        for (int i = 0; i < distanceTable.length - 1; ++i) {

            distanceTable[i] = new Double[i + 1];

            for (int j = 0; j <= i; ++j) {

                double distance = 0.0;

                for (int k = 0; k < DataSet[i + 1].length; ++k) {

                    double difference = Double.parseDouble(DataSet[j][k]) - Double.parseDouble(DataSet[i + 1][k]);

                    distance += difference * difference;

                }

                distanceTable[i][j] = distance;

            }

        }

        /* END WANT TO REPLACE THIS MATRIX CALCULATION WITH PARALLEL STREAM RATHER THAN USE TRADITIONAL ARRAY ARITHMETICS START  */

        } catch ( Exception except ){

            System.out.println ( except );

        }

我宁愿不使用库或类似的东西，我这样做主要是为了了解它是如何工作的。预先非常感谢您。如果你询问数据看起来像：

4,53

5,63

10,59

9,77

13,49

数据处理的输出应如下所示：

[101] <- ((4-5)^2) + ((53-63)^2)

[72, 41] <- ( ((4-10)^2) + ((53-59)^2) ), ( ((5,10)^2) + ((63-59)^2))

[601.0, 212.0, 325.0]

[97.0, 260.0, 109.0, 800.0]

[337.0, 100.0, 109.0, 80.0, 400.0]

Answer 1

我尝试matrixDistance用 来改变distanceTable。尝试将此代码移至不同的方法中，以便可以并行运行它&nbsp; &nbsp; &nbsp; &nbsp; for(int i = 0; i < matrixDistance.length - 1; ++i) {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; distanceTable[i] = new double[i + 1];&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; for(int j = 0; j <= i; ++j) {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; double distance = 0.0;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; for(int k = 0; k < DataSet[i+1].length; ++k) {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; double difference = Double.parseDouble(DataSet[j][k]) - Double.parseDouble(DataSet[i+1][k]);&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; distance += difference * difference;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; distanceTable[i][j] = distance;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }&nbsp; &nbsp; &nbsp; &nbsp; }我根据你的问题创建了这个例子。&nbsp; &nbsp; public void parallel(String file)&nbsp; &nbsp; ....&nbsp; &nbsp; // parsing from csv into matrix 2d Double[][]&nbsp; &nbsp; ....&nbsp; &nbsp; &nbsp; &nbsp; IntStream&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; .range(1, data.length - 1)&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; .parallel()&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; .forEach(i -> {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; add(euclidian.euclidian(Arrays.copyOf(data, i+1)), i);&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; });}这是你的算法的迷你版本。&nbsp; &nbsp; public Double[] euclidian(Double[][] data) {&nbsp; &nbsp; &nbsp; &nbsp; Double[] result = new Double[data.length - 1];&nbsp; &nbsp; &nbsp; &nbsp; for (int i = 0; i < result.length; i++) {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; result[i] =&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Math.pow(data[i][0] - data[data.length - 1][0], 2) +&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Math.pow(data[i][1] - data[data.length - 1][1], 2);&nbsp; &nbsp; &nbsp; &nbsp; }&nbsp; &nbsp; &nbsp; &nbsp; return result;&nbsp; &nbsp; }并且由于并行执行，您需要添加锁定方法以将数据插入distanceTable。&nbsp; &nbsp; private final Object lock = new Object();&nbsp; &nbsp; Double[][] distanceTable;&nbsp; &nbsp; void add(Double[] data, int index){&nbsp; &nbsp; &nbsp; &nbsp; synchronized (lock) {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; distanceTable[index - 1] = data;&nbsp; &nbsp; &nbsp; &nbsp; }&nbsp; &nbsp; }我已经在我的笔记本电脑上测试了它，对于 csv 文件中的 74 行，比较如下（ORI 使用您的代码，PAR 使用我的方法）：java -jar target/stream-example-1.0-SNAPSHOT.jar test.csv&nbsp;#####################ORI read: 59 msORI&nbsp; map: 71 msORI time: 80 ms#####################PAR read: 0 msPAR&nbsp; map: 6 msPAR time: 11 ms希望能帮助到你。

Answer 2

例如，使用Double.parseDouble@Fahim Bagar 提供的代码示例可以轻松消除浪费String[][] DataSetDouble[][] DataSet//String[][] DataSet = Matrix.map(mapping -> mapping.split(",")).toArray(String[][]::new);Double[][] DataSet = Matrix.map(row -> Arrays.stream(row.split(",")).map(Double::parseDouble).toArray(Double[]::new)).toArray(Double[][]::new);然后在循环之外获取局部变量的DataSet[i + 1]各种数组引用：DataSet[j]for (int i = 0; i < distanceTable.length - 1; ++i) {&nbsp; &nbsp; Double[] arriplus1 = new Double[i + 1];&nbsp; &nbsp; Double[] iarr = DataSet[i + 1];&nbsp; &nbsp; for (int j = 0; j <= i; ++j) {&nbsp; &nbsp; &nbsp; &nbsp; double distance = 0.0;&nbsp; &nbsp; &nbsp; &nbsp; Double[] jarr = DataSet[j];&nbsp; &nbsp; &nbsp; &nbsp; for (int k = 0, sz = iarr.length; k < sz; ++k) {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; double difference = jarr[k] - iarr[k];&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; distance += difference * difference;&nbsp; &nbsp; &nbsp; &nbsp; }&nbsp; &nbsp; &nbsp; &nbsp; arriplus1[j] = distance;&nbsp; &nbsp; }&nbsp; &nbsp; distanceTable[i] = arriplus1;}您可以对@Fahim Bagareuclidian方法做同样的事情public Double[] euclidian(Double[][] data) {&nbsp; &nbsp; Double[] result = new Double[data.length - 1];&nbsp; &nbsp; Double[] dL1 = data[data.length - 1];&nbsp; &nbsp; for (int i = 0; i < result.length; i++) {&nbsp; &nbsp; &nbsp; &nbsp; Double[] di = data[i];&nbsp; &nbsp; &nbsp; &nbsp; result[i] = Math.pow(di[0] - dL1[0], 2) + Math.pow(di[1] - dL1[1], 2);&nbsp; &nbsp; }&nbsp; &nbsp; return result;}之后，摆脱Double并使用double将进一步加快/减少内存分配。在 CSV 第 1048 行中，我在每次运行第 10 次时看到这些计时：#####################ORI read: 0 msORI&nbsp; map: 4 msORI time: 14 ms#####################PAR read: 0 msPAR&nbsp; map: 1 msPAR time: 10 ms