Home My Page Projects PaStiX
Summary Activity Forums Lists Docs News Files

Forum: help

Monitor Forum | Start New Thread Start New Thread
RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-10-01 06:38
[forum:147843]
Thanks for the feedback and the porting to homebrew, I'll have a look at that.
I suppose this will also be solve by using CMake ( Mathieu we are waiting for your branch ;) ) because it will decide the right name/links to create.

RE: Test failure [ Reply ]
By: Dominique Orban on 2013-09-30 15:19
[forum:147842]
All seems well and all tests pass with release 4492. I updated the formula: https://gist.github.com/6616136

I think we're ready to submit it to Homebrew and get their comments. I registered on the mailing list so I'll be able to update the formula in the future.

Thanks again for all your help!

RE: Test failure [ Reply ]
By: Dominique Orban on 2013-09-30 18:01
[forum:147841]
Thank you Xavier.

My pull request can be found here: https://github.com/Homebrew/homebrew-science/pull/365

As a last detail, something I hadn't noticed before: the name of the versioned libraries follows the Linux convention, e.g., libmatrix_driver.dylib.5.2. On OSX, the version number should appear before the extension: libmatrix_driver.5.2.dylib.

Also, it's unusual to have the versioned library be a symbolic link to the actual library (e.g., libmatrix_driver.dylib.5.2 -> libmatrix_driver.dylib). It's normally the other way around (libmatrix_driver.dylib -> libmatrix_driver.5.2.dylib). For instance, on my system,

$ ll /usr/lib/libmenu*
/usr/lib/libmenu.5.4.dylib
/usr/lib/libmenu.dylib -> libmenu.5.4.dylib

Your setup is working as it is but it's nonstandard on OSX. Technically, the linked won't pick up versioned libraries if they have the wrong naming convention. I think in your case, things are working because you have the symlinks the other way around.

Perhaps some detail to fix when you have an intern looking for some warmup. :)

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-30 15:04
[forum:147840]
Ok I just updated the release file :

For the mailing list, Pierre Ramet send releases mails to pastix-users@lists.gforge.inria.fr and some more people, you can join it through the list tab on gforge or we can add you manually as well.

XL.

RE: Test failure [ Reply ]
By: Dominique Orban on 2013-09-30 14:19
[forum:147839]
Yes the METIS 5 interface is different. I'll just keep the metis4 dependency.

Not a problem if you update the tarball. Let me know and I'll test the formula again. If all is well, I'll submit it to Homebrew.

Is there a mailing list where I would be updated of such updates to existing and new releases?

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-30 14:11
[forum:147838]
I think providing Scotch 6.0.0 is fine to me, 5.1.12 can give better results in some cases but I suppose that if someone want more tuning he can do install from sources.
If Metis interface as changed in version 5 it won't be supported, we also need to add ParMETIS support...
We found that sdint.h was missing in 5.2.1, I may change the tarball again, is this a problem for you ?

RE: Test failure [ Reply ]
By: Dominique Orban on 2013-09-30 14:07
[forum:147837]
Thank you Xavier. A couple of final questions:

- Is there any advantage to offering a "Scotch 5" option, or should I just always use Scotch 6?

- Is METIS 5 supported?

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-30 13:44
[forum:147836]
I was using 4.8.1 from homebrew

┌─(~)─────────────(lacoste@Cronos-2:s011)─┐
└─(15:43:21)──> which gfortran
/usr/local/bin/gfortran
┌─(~)─────────────(lacoste@Cronos-2:s011)─┐
└─(15:43:23)──> gfortran --version
GNU Fortran (GCC) 4.8.1
Copyright (C) 2013 Free Software Foundation, Inc.

GNU Fortran comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of GNU Fortran
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING

XL.

RE: Test failure [ Reply ]
By: Dominique Orban on 2013-09-30 13:23
[forum:147835]
I understand. Could you let me know what Fortran compiler you were using when you tested the Homebrew formula? Was it the gfortran 4.2.1 that ships with OSX or the gfortran 4.8 that you can install via Homebrew?

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-30 10:02
[forum:147834]
Hello,

We require the libgfortran to build the examples because some of the drivers are written in Fortran. But as the example is in C we generate de binary wih mpicc, thus we have to add -lgfortran or -lifcore.

In fact the -rsa driver uses skitf.f, you should be able to disable it with -DUSE_NOFORTRAN (you can add it in CCTYPES for example).

Maybe we should use the hb_io.c file instead of skitf.f (when I google wreadmtc skitf.f, it seem we are the only ones using this...) but as far as I remember we kept this Fortran file because the hb_io.c had some bugs.... Have to look at it...

XL.



RE: Test failure [ Reply ]
By: Dominique Orban on 2013-09-28 22:30
[forum:147833]
Hi Xavier,

Sorry I have another one here because compiler versions on Homebrew are causing me a bit of trouble. I've been trying to build PaSTiX with only mpicc and mpif90. I removed `libgfortran` from EXTRALIBS. To my surprise, I got error messages saying some symbols related to gfortran were undefined. Is it expected that there is a hard dependence on gfortran?

The following happens during `make install`:

---
mpicc -shared -Wl,-dylib_install_name -Wl,/tmp/pastix_release_4475/src/../install/libmatrix_driver.dylib -o /tmp/pastix_release_4475/src/../install/libmatrix_driver_32bit_mpi_smp_int32_double_real_scotch_i686_mac.dylib matrix_drivers/obj/i686_mac/iohb.o matrix_drivers/obj/i686_mac/mmio.o matrix_drivers/obj/i686_mac/common_drivers.o matrix_drivers/obj/i686_mac/get_options.o matrix_drivers/obj/i686_mac/skitf.o matrix_drivers/obj/i686_mac/read_matrix.o matrix_drivers/obj/i686_mac/rsaread.o matrix_drivers/obj/i686_mac/hbread.o matrix_drivers/obj/i686_mac/mmread.o matrix_drivers/obj/i686_mac/mmdread.o matrix_drivers/obj/i686_mac/petscread.o matrix_drivers/obj/i686_mac/cccread.o matrix_drivers/obj/i686_mac/olafread.o matrix_drivers/obj/i686_mac/chbread.o matrix_drivers/obj/i686_mac/cscdread.o matrix_drivers/obj/i686_mac/peerread.o matrix_drivers/obj/i686_mac/threefilesread.o matrix_drivers/obj/i686_mac/fdupread.o matrix_drivers/obj/i686_mac/laplacian.o matrix_drivers/obj/i686_mac/read_matrix_s.o matrix_drivers/obj/i686_mac/rsaread_s.o matrix_drivers/obj/i686_mac/hbread_s.o matrix_drivers/obj/i686_mac/mmread_s.o matrix_drivers/obj/i686_mac/mmdread_s.o matrix_drivers/obj/i686_mac/petscread_s.o matrix_drivers/obj/i686_mac/cccread_s.o matrix_drivers/obj/i686_mac/olafread_s.o matrix_drivers/obj/i686_mac/chbread_s.o matrix_drivers/obj/i686_mac/cscdread_s.o matrix_drivers/obj/i686_mac/peerread_s.o matrix_drivers/obj/i686_mac/threefilesread_s.o matrix_drivers/obj/i686_mac/fdupread_s.o matrix_drivers/obj/i686_mac/laplacian_s.o matrix_drivers/obj/i686_mac/read_matrix_d.o matrix_drivers/obj/i686_mac/rsaread_d.o matrix_drivers/obj/i686_mac/hbread_d.o matrix_drivers/obj/i686_mac/mmread_d.o matrix_drivers/obj/i686_mac/mmdread_d.o matrix_drivers/obj/i686_mac/petscread_d.o matrix_drivers/obj/i686_mac/cccread_d.o matrix_drivers/obj/i686_mac/olafread_d.o matrix_drivers/obj/i686_mac/chbread_d.o matrix_drivers/obj/i686_mac/cscdread_d.o matrix_drivers/obj/i686_mac/peerread_d.o matrix_drivers/obj/i686_mac/threefilesread_d.o matrix_drivers/obj/i686_mac/fdupread_d.o matrix_drivers/obj/i686_mac/laplacian_d.o matrix_drivers/obj/i686_mac/read_matrix_c.o matrix_drivers/obj/i686_mac/rsaread_c.o matrix_drivers/obj/i686_mac/hbread_c.o matrix_drivers/obj/i686_mac/mmread_c.o matrix_drivers/obj/i686_mac/mmdread_c.o matrix_drivers/obj/i686_mac/petscread_c.o matrix_drivers/obj/i686_mac/cccread_c.o matrix_drivers/obj/i686_mac/olafread_c.o matrix_drivers/obj/i686_mac/chbread_c.o matrix_drivers/obj/i686_mac/cscdread_c.o matrix_drivers/obj/i686_mac/peerread_c.o matrix_drivers/obj/i686_mac/threefilesread_c.o matrix_drivers/obj/i686_mac/fdupread_c.o matrix_drivers/obj/i686_mac/laplacian_c.o matrix_drivers/obj/i686_mac/read_matrix_z.o matrix_drivers/obj/i686_mac/rsaread_z.o matrix_drivers/obj/i686_mac/hbread_z.o matrix_drivers/obj/i686_mac/mmread_z.o matrix_drivers/obj/i686_mac/mmdread_z.o matrix_drivers/obj/i686_mac/petscread_z.o matrix_drivers/obj/i686_mac/cccread_z.o matrix_drivers/obj/i686_mac/olafread_z.o matrix_drivers/obj/i686_mac/chbread_z.o matrix_drivers/obj/i686_mac/cscdread_z.o matrix_drivers/obj/i686_mac/peerread_z.o matrix_drivers/obj/i686_mac/threefilesread_z.o matrix_drivers/obj/i686_mac/fdupread_z.o matrix_drivers/obj/i686_mac/laplacian_z.o matrix_drivers/obj/i686_mac/api_str_to_int.o -lm -L/usr/local/Cellar/scotch/6.0.0/lib -lptscotch -lscotch -lptscotcherrexit -lpthread -lblas -L/tmp/pastix_release_4475/src/../install/ -lpastix
Undefined symbols for architecture x86_64:
"__gfortran_st_close", referenced from:
_readmtc_ in skitf.o
_readmt_c_ in skitf.o
"__gfortran_st_open", referenced from:
_readmtc_ in skitf.o
_readmt_c_ in skitf.o
"__gfortran_st_read", referenced from:
_readmtc_ in skitf.o
_readmt_ in skitf.o
_readmt_c_ in skitf.o
"__gfortran_st_read_done", referenced from:
_readmtc_ in skitf.o
_readmt_ in skitf.o
_readmt_c_ in skitf.o
"__gfortran_st_write", referenced from:
_dump_ in skitf.o
"__gfortran_st_write_done", referenced from:
_dump_ in skitf.o
"__gfortran_transfer_character", referenced from:
_readmtc_ in skitf.o
_readmt_ in skitf.o
_readmt_c_ in skitf.o
"__gfortran_transfer_integer", referenced from:
_readmtc_ in skitf.o
_readmt_ in skitf.o
_readmt_c_ in skitf.o
"__gfortran_transfer_integer_write", referenced from:
_dump_ in skitf.o
"__gfortran_transfer_real", referenced from:
_readmtc_ in skitf.o
_readmt_ in skitf.o
_readmt_c_ in skitf.o
"__gfortran_transfer_real_write", referenced from:
_dump_ in skitf.o
----

Thanks!

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-19 12:59
[forum:147799]
The reentrant issue can be there or not depending on the way the threads are executing.

I suppose there is an issue with flex, I'll see with François.

Thanks for all your time spent on this formula !

RE: Test failure [ Reply ]
By: Dominique Orban on 2013-09-19 12:50
[forum:147798]
Thank you Xavier! I'm now using your latest tar ball and I disabled the patch. All seems well, including the paths returned by `otool -L libpastix.dylib`.

For some reason, the `reentrant` test passes when I run it by hand but not when `brew` runs it. I'm not sure what's going on.

All other tests pass. Thanks a lot for your help! I'll go ahead and submit this formula to Homebrew/Science.

Dominique

ps: is METIS 4 the right version or is METIS 5 also supported?

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-19 11:16
[forum:147797]
With the new 5.2.1 I commited all this pass :

def test
Dir.foreach("#{share}/example/bin") do |example|
next if example =~ /^\./ or example =~ /plot_memory_usage/
# The following tests currently crash.
next if example == 'reentrant'
if example == 'murge-product'
system "#{share}/example/bin/#{example} 100 10 1"
else
if example =~ /murge/
system "#{share}/example/bin/#{example} 100 10"
else
system "#{share}/example/bin/#{example} -lap 100" # 100x100 Laplacian.
end
end
end
ohai 'All test output is in ~/Library/Logs/Homebrew/pastix. Please check.'
end

I updated the tarball also.

XL

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-19 06:32
[forum:147796]
I don't understand, what did fix reentrant/fsimple ?

I'll try your formula, the patch is OK for homebrew but I wanted to add it to 5.2.1 so that you don't have to patch.

XL.

RE: Test failure [ Reply ]
By: Dominique Orban on 2013-09-18 21:47
[forum:147795]
Hi Xavier,

Thanks, `fsimple` runs fine now, as does `reentrant`. The two remaining failures are `isolate_zeros` and `murge_product`.

Here's my current formula: http://git.io/pmtBmA

You may need to disable the patch, depending on what you fixed. I think my Makefile rule was ok since `../install` is not supposed to stick around (is it?) At least with Homebrew, `../install` is destroyed and nothing is ever run from there.

Dominique

RE: Test failure [ Reply ]
By: Nobody on 2013-09-17 16:36
[forum:147794]
If that can be of any help, I'm using Homebrew OpenMPI (version 1.6.5) and built with `--enable-mpi-thread-multiple`.

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-18 10:05
[forum:147790]
To François,

Adding a global mutex seems to solve the issues :

pthread_mutex_lock(&mutex_scotch);
ret = SCOTCH_stratGraphOrder (&stratdat, strat);
pthread_mutex_unlock(&mutex_scotch);

XL.

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-18 09:27
[forum:147789]
Hello Dominique,

For the fsimple.F90 I backported a fix from trunk in 5.2.1 (nnz/check_data were not initialized)

Can you gist your homebrew recipe so that I can tests it ?

XL.

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-17 16:04
[forum:147788]
Maybe an issue with flex ???

http://stackoverflow.com/questions/13940954/flex-giving-fatal-scanner-internal-error-end-of-buffer-missed

"The problem is that flex's default scanner is not reentrant -- it stores a bunch of info (including the current buffer to read from) in global variables, so if you try to have multiple threads scanning stuff at the same time, they'll step all over each other."

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-17 15:53
[forum:147787]
And also with 5.1.4 (Back to Feb. 2011)...
I have to check if it occurs on Linux too or only on MacOS...

Same on linux :

fatal flex scanner internal error--end of buffer missed
(0): ERROR: stratParserParse: invalid strategy string, before "0.7,cpr=n{sep=/(vert>120)?m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}}|m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}};,ole=f{cmin=0,cmax=100000,frat=0.0},ose=g},unc=n{sep=/(vert>120)?(m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}})|m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}};,ole=f{cmin=15,cmax=100000,frat=0.08},ose=g}}"

XL.

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-17 15:38
[forum:147786]
The error occurs also with scotch 5.1.12, it may also come from PaStiX before 5.2... erf...

XL.

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-17 15:22
[forum:147785]
I can also have a stack when I get a crash :

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000
[Switching to process 71072 thread 0x1b03]
yy_init_buffer [inlined] () at :1534
1534 lex.yy.c: No such file or directory.
in lex.yy.c
(gdb) bt
#0 yy_init_buffer [inlined] () at :1534
#1 0x00000001002a7d3e in yyrestart (input_file=0x0) at parser_ll.c:1427
#2 0x00000001002a7e18 in _SCOTCHstratParserSelect [inlined] () at /Users/lacoste/INRIA/scotch_6.0.0rc17/src/libscotch/parser_ll.l:198
#3 0x00000001002a7e18 in _SCOTCHstratParserInit (string=0x101345750 "c{rat=0.7,cpr=n{sep=/(vert>120)?m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}}|m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}};,ole=f{cmin=0,cmax=100000,frat=0.08},ose=g},unc=n{sep=/(vert>120)?(m"...) at parser_ll.c:1427
#4 0x00000001002a9b81 in _SCOTCHstratParserParse (strattab=0x1002e4d60, string=0x0) at parser_yy.y:782
#5 0x00000001002a5139 in SCOTCH_stratGraphOrder (stratptr=0x101345748, string=0x58 <Address 0x58 out of bounds>) at library_graph_order.c:473
#6 0x000000010008650b in pastix_task_scotch ()
#7 0x0000000100089dbe in pastix ()
#8 0x0000000100001bdc in solve_smp ()
#9 0x00007fff933a98bf in _pthread_start ()
#10 0x00007fff933acb75 in thread_start ()

RE: Test failure [ Reply ]
By: Xavier Lacoste on 2013-09-17 14:54
[forum:147784]
Hello François,

I though that there was something not reentrant with Scotch because I got errors with the reentrant test of PaStiX inside Scotch (Calls 2 PaStiX in 2 threads).

ERROR: stratParserParse: invalid strategy string, before "at=0.7,cpr=n{sep=/(vert>120)?m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}}|m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}};,ole=f{cmin=0,cmax=100000,frat=0.0},ose=g},unc=n{sep=/(vert>120)?(m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}})|m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}};,ole=f{cmin=15,cmax=100000,frat=0.08},ose=g}}"ERROR:
stratParserParse: invalid method name "{", before "at=0.7,cpr=n{sep=/(vert>120)?m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}}|m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}};,ole=f{cmin=0,cmax=100000,frat=0.0},ose=g},unc=n{sep=/(vert>120)?(m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}})|m{rat=0.8,vert=100,low=h{pass=10},asc=f{bal=0.2}};,ole=f{cmin=15,cmax=100000,frat=0.08},ose=g}}"

The error can change a bit...

There is no error when I run with valgrind because it runs sequentially inside valgrind...

The error may also be inside PaStiX, before calling Scotch.

I asked the valgrind output for the other example crashing which I can't reproduce, I don't know if it's the one Dominique gave but you are right, there is too many errors in MPI_Init_thread to get the real errors in PaStiX.

XL.

RE: Test failure [ Reply ]
By: Francois PELLEGRINI on 2013-09-17 14:30
[forum:147782]
Hello all,

Regarding Scotch 6.0.0, the trunk and the tgz are the same.

All the errors I saw/overlooked in the (long) valgrind files seem to relate to the initialization of OpenMPI (through orte_init () and its descendants).

What makes you think that there might be something not reentrant in Scotch ?
Most routines are, but I may have overlooked something.

f.p.

Older Messages