Home My Page Projects PaStiX
Summary Activity Forums Lists Docs News Files

Forum: help

Monitor Forum | Start New Thread Start New Thread
RE: PaStiX ordering and analysis phase [ Reply ]
By: Xavier Lacoste on 2013-05-15 11:23
[forum:114217]
Hi Sameer,

Else you can also send me a matrix in a MatrixMarket Format if it is easier.

Thanks for the export line :)

You can attach data to the forum or send it to xavier.lacoste _at_ inria.fr , as you want.

XL.

RE: PaStiX ordering and analysis phase [ Reply ]
By: Sameer Agarwal on 2013-05-14 17:01
[forum:114216]
Hi Xavier,
Thanks for replying.

Let me construct a stand alone version of the code and data and send it to you. Since building and running Ceres maybe too much work, and obscures the issue.

As for getting the threading performance, yes this required some work. The problem is that the underlying vecLib blas/lapack implementation is already threaded and the threading in pastix conflicts with it

Forcing the blas to run single threaded by setting

export VECLIB_MAXIMUM_THREADS=1

did the trick. That said, this is not really optimal and it would be nice to have pastix account for and control the threading behaviour of blas when it is figuring out the scheduling. But I do not think there is a robust cross platform way of doing it, since BLAS libraries do not have a standard way of controlling threading.

Would you prefer the code and data sent via private email or here on the forum?

Sameer

RE: PaStiX ordering and analysis phase [ Reply ]
By: Xavier Lacoste on 2013-05-14 07:39
[forum:114047]
Hello,

First, thanks for your feedback.

csc_symgraph_int is performed when the matrix is symmetric because preprocessing steps require a symmetric graph (By Scotch when using scotch or kass() when using other ordering).

It is not runned with non symmetric matrix because the graph is already supposed to be symmetric (yes, it's can seem quite odd...)

If you want I can try to run your code to try to understand why this step is so long.

Did you manage to have some scaling with threads ? (I remember it was not easy on MacOS, I had some issues with the BLAS, can you remember me how to use single threaded BLAS correctly ? )

Your pastix-conf seems correct to me.

XL.






PaStiX ordering and analysis phase [ Reply ]
By: Sameer Agarwal on 2013-05-10 14:54
[forum:114042]
Hello,

First of all thank you for making a PaStiX available as an open source project. It is great to have such a comprehensive sparse matrix factorization library with support for threading.

I am one of the developers of Ceres Solver (https://code.google.com/p/ceres-solver/) and I am interested in using PaStiX for solving large sparse Cholesky factorization problems. I have a few questions in this regards.

I am using pastix in multithreaded mode without mpi.

1. The ordering phase of pastix seems rather slow. Pastix is an order of magnitude slower when using scotch than CHOLMOD + AMD.

2. But the slow down is not just due to SCOTCH. Even when I disable SCOTCH and instead pass an already ordered (using AMD) matrix, things take a lot of time.

The principal source of this seems to be csc_symgraph_int which is symmetrizing the graph. What is odd about this function call is that the matrix that I supplied to pastix was already symmetric upper triangular, and this function seems to be working under the assumption that its dealing with a general unsymmetric matrix. For the case of my matrix, it should be trivial to symmetrize this graph.

To give you an example of the difference in time. For a 6696 x 6696 matrix

pastix + scotch : 61 seconds
pastix + amd (via API_ORDER_PERSONAL) : 19 seconds
cholmod + amd: 3.6 seconds

All of these numbers are on a macbook with the vecLib BLAS running single threaded.
scotch version 6.0.0 and SuiteSparse 4.2.1 were installed via homebrew.

Pastix release 4030 was used, which I compiled myself. Here is the output

pastix-conf_64bit_nompi_smp_int32_double_real_scotch_i686_mac.sh
Librairies : -


L/Users/sameeragarwal/Downloads/pastix_release_4030/src/../install -lpastix -L/usr/local/homebrew/Cellar/gfortran/4.7.2/gfortran/lib -lgfortran -lm -L/usr/local/homebrew/lib -lscotch -lscotcherrexit -L/usr/local/homebrew/lib -lscotch -lpthread
Librairies with murge : -DMURGE_MESSAGE="To use Murge interface compile with -DDISTRIBUTED"
Incs : -I/Users/sameeragarwal/Downloads/pastix_release_4030/src/../install
C Compiler : cc
C++ Compiler : mpic++
C Compiler options : -O3 -Wall -DINTSIZE32 -DFORCE_NOMPI -DFORCE_NO_CUDA -DPASTIX_DYNSCHED -I/usr/local/homebrew/include -DWITH_SCOTCH -DVERSION='exported' -DX_ARCHi686_mac -DDOF_CONSTANT
Fortran Compiler : gfortran
Fortran Compiler options :
C Linker : cc
Fortran Linker : gfortran
Options : -DINTSIZE32 -DFORCE_NOMPI -DPREC_DOUBLE -DFORCE_NO_CUDA -DPASTIX_DYNSCHED -I/usr/local/homebrew/include -DWITH_SCOTCH
Version : exported
Blas :

I am happy to show you my code and data and/or pastix logs if that will help.

Thank you,
Sameer