Difference between revisions of "Web Application Hosting"

From UFRC
Jump to navigation Jump to search
 
(32 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[Category:Services]][[Category:PubApps]][[Category:Web]]
+
[[Category:Services]][[Category:PubApps]]
=Purpose=
+
The mission of UF Research Computing is to support research done on HiPerGator, whether with computational resources, applications, support, or additional services like PubApps.  HiPerGator Web Application Hosting a.k.a. 'PubApps' is not intended to be a generic web application hosting service. Use [https://webservices.it.ufl.edu/terminalfour/hosting/ UFIT T4 CMS Hosting], [https://hosting.it.ufl.edu/services/apache-hosting/ UFIT Apache Hosting], or [https://hosting.it.ufl.edu/services/virtual-machine-hosting/ UFIT VM Hosting]] for document or web application and service hosting unrelated to HiPerGator or UF Research Computing. PubApps started as shared infrastructure to enable 'public' web applications with significant computational and database needs and no user access restrictions to showcase existing research it has evolved to add secured Virtual machine based web applications, APIs, and services for internal use by UF research groups or for collaborations.  
HiPerGator Web Application Hosting a.k.a. 'PubApps' is not intended to be a generic web application hosting service. Use [https://webservices.it.ufl.edu/terminalfour/hosting/ UFIT T4 CMS Hosting], [https://hosting.it.ufl.edu/services/apache-hosting/ UFIT Apache Hosting], or [https://hosting.it.ufl.edu/services/virtual-machine-hosting/ UFIT VM Hosting]] for document or generic web application and service hosting.
 
  
PubApps started as shared infrastructure to enable 'public' web applications with significant computational and database needs and no user access restrictions to showcase existing research it has evolved to add secured Virtual machine based web applications, APIs, and services for internal use by UF research groups or for collaborations.
+
The purpose of PubApps is to provide the kind of 'long-tail' services and infrastructure that cannot be obtained or set up elsewhere or requires access to data/computational resources on HiPerGator and so on. It can give UF Research Computing customers a competitive edge for top-tier research and presentation of research, with such needs as: significant computational and database resources and/or application support, access to data on large parallel filesystems, or setting up services connected to data-producing hardware on premises at the University to feed into workflows running on HiPerGator.
  
The purpose of PubApps is to provide the kind of 'long-tail' services and infrastructure that cannot be obtained or set up elsewhere or requires access to data/computational resources on HiPerGator and so on that can give UF Research Computing customers a competitive edge for top-tier research and presentation of research. Some examples of such needs include significant computational and database resources and/or application support, access to data on large parallel filesystems, setting up services connected to data-producing hardware on premises at the University to feed into workflows running on HiPerGator, creating interfaces for more efficient data processing for research groups.
+
==Architecture==
 +
[[Image:Rc_web_app_hosting.svg|800px]]
  
=Provisioning and Options=
+
==Purchasing==
* Please reach out to use via [https://support.rc.ufl.edu/enter_bug.cgi Support System] or one of the options listed in [https://www.rc.ufl.edu/help/remote-support/ RC Remote Support] beforehand, so we could discuss your project and determine whether it's appropriate for PubApps or if we could refer you to one of the UFIT hosting solutions or external resources.
+
A basic PubApps instance with up to 2 virtual CPU cores, 16GB of memory, and 1TB of disk space can be purchased via the [https://www.rc.ufl.edu/get-started/purchase-allocation/ HiPerGator Service Purchase] form for $300 per year. If additional resources are required CPU/memory (NCU), storage, and GPUs can be purchased in the same manner as HPG resources are purchased.
* [https://www.rc.ufl.edu/access/purchase-request/hpg-service/ Submit a purchase request] for a 'PubApps Service' instance.
+
 
 +
==Provisioning and Options==
 +
* Please reach out to use via [https://support.rc.ufl.edu/enter_bug.cgi Support System] or one of the options listed in [https://www.rc.ufl.edu/help/remote-support/ RC Remote Support] beforehand, so we could discuss your project and determine whether it's appropriate for PubApps or if we could refer you to one of the UFIT hosting solutions or external resources. A conversation with the UFRC staff is required to determine whether a standalone private VM or resources in a shared VM will be provisioned for a basic instance.
 +
 
 +
* [https://gravity.rc.ufl.edu/access/purchase-request/hpg-service/ Submit a purchase form] for a 'PubApps Service' instance as described above.
 
* Open a [https://support.rc.ufl.edu/enter_bug.cgi support request] for configuration of your PubApps VM or Shared instance. We'll set up infrastructure and dependencies necessary to start running or developing your public or internal application.
 
* Open a [https://support.rc.ufl.edu/enter_bug.cgi support request] for configuration of your PubApps VM or Shared instance. We'll set up infrastructure and dependencies necessary to start running or developing your public or internal application.
  
We provide an SSL Certificate and an .rc.ufl.edu sub-domain for your site for free. Please talk to us if you need to migrate an external site e.g. something from .org domain to UFRC.
+
''Note: We provide an SSL Certificate and an .rc.ufl.edu sub-domain for your site for free. Please talk to us if you need to migrate an external site e.g. something from .org domain to UFRC.'''
  
 
Since we proxy all applications through our load balancer we can configure an authentication/authorization layer between your entire web application or some parts of it and the clients accessing it based on the same UFRC Single-Signon we use for all UFRC websites and web applications. The options for protection include selected HiPerGator group(s), all HiPerGator users, or all UF Gatorlink accounts. Of course, basic http auth (preset username/password) can also be configured.
 
Since we proxy all applications through our load balancer we can configure an authentication/authorization layer between your entire web application or some parts of it and the clients accessing it based on the same UFRC Single-Signon we use for all UFRC websites and web applications. The options for protection include selected HiPerGator group(s), all HiPerGator users, or all UF Gatorlink accounts. Of course, basic http auth (preset username/password) can also be configured.
  
=PubApps Virtual Hosting Infrastructure=
+
General questions we are going to ask before setting up an instance:
PubApps Virtual Hosting is an evolution of the original PubApps Shared Hosting. Using a standalone VM (Virtual Machine) is a good solution for when the project needs dedicated resources, has more stringent security requirements, requires access to Blue or Orange storage, and may be running applications that are created to serve their every component (web application itself, database server, other services) from a single machine, etc.
+
 
 +
* Purpose and the intended audience for the web application(s) you are going to run.
 +
* Access restrictions:
 +
** '''Public Open''': Visible outside of the UF network. There are no access restrictions or authentication is used only for administrative paths.
 +
** '''Public Restricted''': Visible outside of the UF network. Authentication is required for access to the application (Apache basic http auth or UFRC single signon with the following possible restrictions: All UF Gatorlink account holders, all HPG users, particular HPG group(s)).
 +
** '''Internal''': Visible only on the UF network or via UF VPN. Authentication is required for the entire application.
 +
* Computational, storage, and database resource requirements.
 +
* Type(s) of the backend database(s) used if any.
 +
* Can your code use an external compute node i.e. submit SLURM jobs to run heavy duty analyses, have an external daemon that can be run on a compute node, or will all analyses run within the application on the webserver?
 +
* Do you need HiPerGator filesystem access (only for internal applications in standalone VMs with defined resources and not the larger shared PubApps webservers because of security considerations).
 +
* Suggested subdomain e.g. myresearchapp.rc.ufl.edu for the requests to be proxied to the application running on a pubapps server.
 +
* HiPerGator username of the developmer/maintainer? to set up ssh access to the pubapps server. We also suggest setting up CI (continuous integration), so code changes in an external git repository get applied to the application.
 +
* Application dependencies/installation documentation and the application source code if you require our help for the initial setup.
 +
 
 
==Virtual Machines==
 
==Virtual Machines==
A PubApps VM has 1 vCPU, 4GB of RAM, and 40GB of disk space.
+
Many web applications/frameworks will happily co-exist with other applications on a large shared host with the only difference being the port that the application backend listens on, which will be reverse proxied to the FQDN (yourapp.rc.ufl.edu). Those applications generally run on pubweb or privweb webservers or pubcontainer* nodes on the public side. However, some applications do not play well with others and require a dedicated VM.
 +
 
 +
Virtual Machine hosting is an available option for applications that don't work well on large shared servers. Using a standalone VM (Virtual Machine) is a good solution for when the project needs dedicated resources, has more stringent security requirements, requires access to Blue or Orange storage, and may be running applications that are created to serve their every component (web application itself, database server, other services) from a single machine, etc.<br>
 +
 
 +
Both the public and the private side of the webapp hosting can have VMs.
 +
 
 +
A basic VM that comes with a pubapps instance purchase has 2 vCPUs, 16GB of RAM, and 40GB of disk space in the root partition besides the /pubapps or /privapps storage.
  
==Filesystem Access==
+
===Filesystem Access===
VMs can be set up with access to /blue/GROUP and /orange/GROUP by a special internal service user with membership in the GROUP setting up the project.
+
PubApps VMs have access to /pubapps/$PROJECT storage.
==Operating System==
 
VMs run RedHat Enterprise Linux as KVM instances.
 
  
=PubApps Shared Hosting Infrastructure=
+
PrivApps VMs can access /blue/GROUP and /orange/GROUP by using a service user with membership in the HiPerGator GROUP.
==Web Servers==
 
PubApps Shared architecture has two webservers and all applications are reverse-proxied by UFRC Load Balancer for redundancy.
 
==Database Server==
 
PubApps Shared includes a database server with MariaDB, PostgreSQL, and MongoDB instances, so the resources on web servers and VMs would not be used for i/o-intensive database operations.
 
==File Server==
 
PubApps Shared includes an NFS file server where each project (group) gets their own directory tree as /data/GROUP.
 
==Computational Resources==
 
PubApps Shared includes compute nodes that allow web applications to perform computational work without overloading web servers. See
 
[[Pubapps_slurm-drmaa|Using PubApps Computational Resources]] for details on using slurm-drmaa python module to access those resources, which can also be accessed via standard SLURM jobs from the web servers similar to how jobs are submitted to HiPerGator.
 
==Operating System==
 
PubApps Shared runs on bare metal RedHat Enterprise Linux instances.
 
  
==Infrastructure Diagram==
+
===Operating System===
An overview of PubApps Shared infrastructure:
+
All RCVMs run RedHat Enterprise Linux 8 as KVM instances.
  
[[File:Pubapps_diagram_new.png|PubApps|800px]]
+
==PubApps Shared Hosting Infrastructure==
 +
For more information about PubApps, you can view the [[Shared Hosting Infrastructure]] page.

Latest revision as of 15:43, 3 April 2024

The mission of UF Research Computing is to support research done on HiPerGator, whether with computational resources, applications, support, or additional services like PubApps. HiPerGator Web Application Hosting a.k.a. 'PubApps' is not intended to be a generic web application hosting service. Use UFIT T4 CMS Hosting, UFIT Apache Hosting, or UFIT VM Hosting] for document or web application and service hosting unrelated to HiPerGator or UF Research Computing. PubApps started as shared infrastructure to enable 'public' web applications with significant computational and database needs and no user access restrictions to showcase existing research it has evolved to add secured Virtual machine based web applications, APIs, and services for internal use by UF research groups or for collaborations.

The purpose of PubApps is to provide the kind of 'long-tail' services and infrastructure that cannot be obtained or set up elsewhere or requires access to data/computational resources on HiPerGator and so on. It can give UF Research Computing customers a competitive edge for top-tier research and presentation of research, with such needs as: significant computational and database resources and/or application support, access to data on large parallel filesystems, or setting up services connected to data-producing hardware on premises at the University to feed into workflows running on HiPerGator.

Architecture

Rc web app hosting.svg

Purchasing

A basic PubApps instance with up to 2 virtual CPU cores, 16GB of memory, and 1TB of disk space can be purchased via the HiPerGator Service Purchase form for $300 per year. If additional resources are required CPU/memory (NCU), storage, and GPUs can be purchased in the same manner as HPG resources are purchased.

Provisioning and Options

  • Please reach out to use via Support System or one of the options listed in RC Remote Support beforehand, so we could discuss your project and determine whether it's appropriate for PubApps or if we could refer you to one of the UFIT hosting solutions or external resources. A conversation with the UFRC staff is required to determine whether a standalone private VM or resources in a shared VM will be provisioned for a basic instance.
  • Submit a purchase form for a 'PubApps Service' instance as described above.
  • Open a support request for configuration of your PubApps VM or Shared instance. We'll set up infrastructure and dependencies necessary to start running or developing your public or internal application.

Note: We provide an SSL Certificate and an .rc.ufl.edu sub-domain for your site for free. Please talk to us if you need to migrate an external site e.g. something from .org domain to UFRC.'

Since we proxy all applications through our load balancer we can configure an authentication/authorization layer between your entire web application or some parts of it and the clients accessing it based on the same UFRC Single-Signon we use for all UFRC websites and web applications. The options for protection include selected HiPerGator group(s), all HiPerGator users, or all UF Gatorlink accounts. Of course, basic http auth (preset username/password) can also be configured.

General questions we are going to ask before setting up an instance:

  • Purpose and the intended audience for the web application(s) you are going to run.
  • Access restrictions:
    • Public Open: Visible outside of the UF network. There are no access restrictions or authentication is used only for administrative paths.
    • Public Restricted: Visible outside of the UF network. Authentication is required for access to the application (Apache basic http auth or UFRC single signon with the following possible restrictions: All UF Gatorlink account holders, all HPG users, particular HPG group(s)).
    • Internal: Visible only on the UF network or via UF VPN. Authentication is required for the entire application.
  • Computational, storage, and database resource requirements.
  • Type(s) of the backend database(s) used if any.
  • Can your code use an external compute node i.e. submit SLURM jobs to run heavy duty analyses, have an external daemon that can be run on a compute node, or will all analyses run within the application on the webserver?
  • Do you need HiPerGator filesystem access (only for internal applications in standalone VMs with defined resources and not the larger shared PubApps webservers because of security considerations).
  • Suggested subdomain e.g. myresearchapp.rc.ufl.edu for the requests to be proxied to the application running on a pubapps server.
  • HiPerGator username of the developmer/maintainer? to set up ssh access to the pubapps server. We also suggest setting up CI (continuous integration), so code changes in an external git repository get applied to the application.
  • Application dependencies/installation documentation and the application source code if you require our help for the initial setup.

Virtual Machines

Many web applications/frameworks will happily co-exist with other applications on a large shared host with the only difference being the port that the application backend listens on, which will be reverse proxied to the FQDN (yourapp.rc.ufl.edu). Those applications generally run on pubweb or privweb webservers or pubcontainer* nodes on the public side. However, some applications do not play well with others and require a dedicated VM.

Virtual Machine hosting is an available option for applications that don't work well on large shared servers. Using a standalone VM (Virtual Machine) is a good solution for when the project needs dedicated resources, has more stringent security requirements, requires access to Blue or Orange storage, and may be running applications that are created to serve their every component (web application itself, database server, other services) from a single machine, etc.

Both the public and the private side of the webapp hosting can have VMs.

A basic VM that comes with a pubapps instance purchase has 2 vCPUs, 16GB of RAM, and 40GB of disk space in the root partition besides the /pubapps or /privapps storage.

Filesystem Access

PubApps VMs have access to /pubapps/$PROJECT storage.

PrivApps VMs can access /blue/GROUP and /orange/GROUP by using a service user with membership in the HiPerGator GROUP.

Operating System

All RCVMs run RedHat Enterprise Linux 8 as KVM instances.

PubApps Shared Hosting Infrastructure

For more information about PubApps, you can view the Shared Hosting Infrastructure page.