Difference between revisions of "Sharing Within A Cluster"

From UFRC
Jump to navigation Jump to search
 
(15 intermediate revisions by 7 users not shown)
Line 1: Line 1:
[[Category:Biology]]
+
[[Category:Data]][[Category:Transfer Data]]
 +
{{Obsolete|data_transfer/sharing_cluster}}
 +
{|align=right
 +
  |__TOC__
 +
  |}
 +
Allow users to share files between themselves on the cluster. For file transfer outside of the cluster see [https://help.rc.ufl.edu/doc/Transfer_Data Transfer Data]
  
For groups that want several people to work on the same data it may be useful to read the following information:
+
See also: [https://help.rc.ufl.edu/doc/Storage#Shared_Work_and_Storage_Management Shared Work and Storage Management]
 +
 
 +
Note that if making a copy of the data is sufficient the easiest approach is to set up a [[Globus]] collection on a directory and give the other party read access to the collection.
 +
 
 +
==Sharing to the same group==
 +
* Note that all groups should have a "share" directory within their storage trees. This is the most appropriate way to share data and software among group members.
 +
 
 +
For a group that wants several people to work on the same data it may be useful to use a combination of directory and file permissions and 'umask' settings that affect all newly created files.
 +
 
 +
* To make the data accessible to all group members, but not any other users outside of the group add the following line to the shell initialization script <code>'~/.bashrc'</code> in each group member's home directory:
 +
umask 007
 +
That umask setting will cause every newly created file to have "-rw-rw----" permissions and every new directory to have "-rwxrwx---" permissions giving full access to the user and the group, but not to anyone else.
  
* To make the data accessible to all group members, but not to the users outside of the group edit the <code>'.bashrc'</code> file in each group member's home directory and add the following line to it:
 
umask 0007
 
After that every newly created file will have "-rw-rw----" permissions and every new directory will have "-rwxrwx---" permissions giving full access to the user and the group, but not to anyone else.
 
 
* To change permissions on an existing directory tree to correspond to the above set change to the directory above the one you'd like to start from and run
 
* To change permissions on an existing directory tree to correspond to the above set change to the directory above the one you'd like to start from and run
 
  find DIRECTORY -type d -exec chmod 770 {} \;
 
  find DIRECTORY -type d -exec chmod 770 {} \;
 
  find DIRECTORY -type f -exec chmod 660 {} \;
 
  find DIRECTORY -type f -exec chmod 660 {} \;
where 'DIRECTORY' is the name of the topmost directory in the tree you want to change permissions on. Note that the user who runs the above commands must be the current owner of the directory tree. The commands may take a while to run if the directory tree is large.
+
where 'DIRECTORY' is the name of the topmost directory in the tree you want to change permissions on. Note that the user who runs the above commands must be the current owner of the directory tree. The commands may take a while to run if the directory tree is large. If the directory tree has mixed ownership open a [https://support.rc.ufl.edu support request] and we'll help.
 +
 
 +
==Sharing to different groups==
 +
===Sharing Data===
 +
For data that doesn't change very frequently the easiest method is to share a directory tree with [[Globus]] by creating a shared collection and giving read, write, or read/write permissions to the user(s) from another group. They will be able to transfer/sync the files to their own directory tree.
 +
===Collaborative Work===
 +
====Project Group====
 +
If members of multiple groups need to work on a project together the sponsor(s) of the involved group(s) can allocate some of their Blue Storage investments to create a new project directory tree in /blue. Open a [https://support.rc.ufl.edu support request] and we'll
 +
* Set up a new project group
 +
* Allocate a share of blue storage investment(s) to the project group.
 +
* Add project members from all involved groups to the project group.
 +
Note that allocating computational resources to the project group is unnecessary in most cases as all project members will still be able to use their own groups' computational resources for analyses.
 +
====Secondary Group====
 +
If the hosting sponsor is willing a user or a sponsor can [https://support.rc.ufl.edu open a support request] to request adding a secondary user to a group. If a request is opened by a user we'll ask the hosting group's sponsor for confirmation before making changes.
 +
 
 +
'''Note:''' that user will have the same level of access to storage and computational resources of their secondary group as the primary users of that group.
 +
 
 +
===Custom Access===
 +
If it is desired to give a certain person or persons access to some directories or files in another group's storage and the above options are not acceptable it may be possible to set up a filesystem ACL i.e. additional custom permissions set on top of Linux filesystem permissions. There are many pitfalls and limitations of this approach, so we don't recommend it as the first choice. However, on request and the responsibility of the group's sponsor we can help the group determine and set ACL or ACLs. See [[ACLs]] for more details.

Latest revision as of 16:58, 23 October 2024

This page is obsolete. It is being retained for archival purposes. The current version of this page can be found at https://docs.rc.ufl.edu/data_transfer/sharing_cluster

Allow users to share files between themselves on the cluster. For file transfer outside of the cluster see Transfer Data

See also: Shared Work and Storage Management

Note that if making a copy of the data is sufficient the easiest approach is to set up a Globus collection on a directory and give the other party read access to the collection.

Sharing to the same group

  • Note that all groups should have a "share" directory within their storage trees. This is the most appropriate way to share data and software among group members.

For a group that wants several people to work on the same data it may be useful to use a combination of directory and file permissions and 'umask' settings that affect all newly created files.

  • To make the data accessible to all group members, but not any other users outside of the group add the following line to the shell initialization script '~/.bashrc' in each group member's home directory:
umask 007

That umask setting will cause every newly created file to have "-rw-rw----" permissions and every new directory to have "-rwxrwx---" permissions giving full access to the user and the group, but not to anyone else.

  • To change permissions on an existing directory tree to correspond to the above set change to the directory above the one you'd like to start from and run
find DIRECTORY -type d -exec chmod 770 {} \;
find DIRECTORY -type f -exec chmod 660 {} \;

where 'DIRECTORY' is the name of the topmost directory in the tree you want to change permissions on. Note that the user who runs the above commands must be the current owner of the directory tree. The commands may take a while to run if the directory tree is large. If the directory tree has mixed ownership open a support request and we'll help.

Sharing to different groups

Sharing Data

For data that doesn't change very frequently the easiest method is to share a directory tree with Globus by creating a shared collection and giving read, write, or read/write permissions to the user(s) from another group. They will be able to transfer/sync the files to their own directory tree.

Collaborative Work

Project Group

If members of multiple groups need to work on a project together the sponsor(s) of the involved group(s) can allocate some of their Blue Storage investments to create a new project directory tree in /blue. Open a support request and we'll

  • Set up a new project group
  • Allocate a share of blue storage investment(s) to the project group.
  • Add project members from all involved groups to the project group.

Note that allocating computational resources to the project group is unnecessary in most cases as all project members will still be able to use their own groups' computational resources for analyses.

Secondary Group

If the hosting sponsor is willing a user or a sponsor can open a support request to request adding a secondary user to a group. If a request is opened by a user we'll ask the hosting group's sponsor for confirmation before making changes.

Note: that user will have the same level of access to storage and computational resources of their secondary group as the primary users of that group.

Custom Access

If it is desired to give a certain person or persons access to some directories or files in another group's storage and the above options are not acceptable it may be possible to set up a filesystem ACL i.e. additional custom permissions set on top of Linux filesystem permissions. There are many pitfalls and limitations of this approach, so we don't recommend it as the first choice. However, on request and the responsibility of the group's sponsor we can help the group determine and set ACL or ACLs. See ACLs for more details.