Home My Page Projects Site Admin
Summary Activity Tracker Lists Docs News SCM

[#20687] rare race condition at project creation

2016-08-10 09:16
Submitted by:
Matthieu Imbert (mimbert)
Assigned to:
Matthieu Imbert (mimbert)
rare race condition at project creation

Detailed description
once in a while, for a newly created project, the source control directory do not get the right unix permissions.

For example, see ticket #36507 https://support.inria.fr/Ticket/Display.html?id=36507

project pstracker registered 2016-04-12 13:38

on scm.gforge.inria.fr, directory /gitroot/pstracker owner is root:root instead of root:pstracker_scmro

systasksd log:

*PRD* root@forge-scm2:~# l /var/log/fusionforge/systasksd.stderr.1
-rw-r----- 1 root adm 1093 Apr 12 14:17 /var/log/fusionforge/systasksd.stderr.1

*PRD* root@forge-scm2:~# cat /var/log/fusionforge/systasksd.stderr.1
chgrp: invalid group: ‘pstracker_scmro’
PHP Warning: chmod(): Permission denied in /usr/share/fusionforge/plugins/scmgit/common/GitPlugin.class.php on line 403
sh: 1: cd: can't cd to /gitroot/pstracker
fatal: repository '/gitroot/pstracker/pstracker.git' does not exist
fatal: Not a git repository: '/gitroot/pstracker/users/yhua.git'
fatal: unable to access '/gitroot/pstracker/users/yhua.git/config': Permission denied
PHP Warning: fopen(/gitroot/pstracker/users/yhua.git/hooks/post-update): failed to open stream: Permission denied in /usr/share/fusionforge/plugins/scmgit/common/GitPlugin.class.php on line 414
PHP Warning: fwrite() expects parameter 1 to be resource, boolean given in /usr/share/fusionforge/plugins/scmgit/common/GitPlugin.class.php on line 415
PHP Warning: fclose() expects parameter 1 to be resource, boolean given in /usr/share/fusionforge/plugins/scmgit/common/GitPlugin.class.php on line 416
sh: 1: cannot create /gitroot/pstracker/users/yhua.git/description: Permission denied
chmod: cannot access ‘/gitroot/pstracker/users/yhua.git’: Permission denied

previous fixes was 1c0f2ceda34d746a4db6ae9e46e2562a620316bd https://scm.fusionforge.org/anonscm/gitweb?p=fusionforge/fusionforge.git;a=commitdiff;h=1c0f2ceda34d746a4db6ae9e46e2562a620316bd but it does not seem to be sufficient
Message  ↓
Date: 2016-08-23 14:22
Sender: Matthieu Imbert


I tried hard to reproduce the bug in a controlled way on the qualification platform, with a selenium script registering several projects sequentially and in parallel.
I attach this script test-register-projects.py
I was almost not able to reproduce the bug (but instead was able to crash the qualif platform several times and leave its database in a bad condition one time :-/ )

Function is_group_active_nocache in systasksd ensures that unix group is ok before proceeding to repo creation.
Repo creation is performed by a call to cron create_scm_repos.php which takes no argument and iterates all tasks SCM_REPO in state WIP.

I initially thought that maybe the race condition would be that:
1 - systasksd interates over a task SCM_REPO in state TODO, for a project X
2 - systasksd waits for unix groups of X to be active
3 - systasksd invokes cron create_scm_repos.php to create repo for project X
4 - at the same time, another project Y is registered
5 - create_scm_repos.php iterates over X and Y while Y unix groups aren't ready yet.
But this cannot happen as task SCM_REPO for project Y would not be in state WIP at this point, and would not be iterated over by create_scm_repos.php

Thus, the patch I prepared to ensure that unix groups are active in create_scm_repos.php is *not* relevant (and would introduce subtle bugs risks)/

For now the best decision seems to do nothing as I consumed quite a lot of time on this, with no result. When this problem happens and a ticket is submitted, we have to manually fix the scm repo.

Size Name Date By Download
4 KiBtest-register-projects.py2016-08-23 14:22mimberttest-register-projects.py
Field Old Value Date By
File Added5817: test-register-projects.py2016-08-23 14:22mimbert
assigned_tonone2016-08-10 13:26mimbert