Cyber Infrastructure Plan 2023-2027

Florida Atlantic University Cyber infrastructure Plan

Executive Summary

As part of the strategy to make Florida Atlantic University a top-tier research university, the Office of Information Technology (OIT) strives to provide the experience and technology to support cutting-edge education and innovative scientific research. FAU plans to enhance and build the IT infrastructure across numerous campuses in support of these goals.

Infrastructure Highlights

• Dedicated research computing data center on the Jupiter campus

• 100 Gb/s link to Florida Lambda Rail in Boca Raton

• 10 Gb/s link to Florida Lambda Rail in Jupiter

• 10 Gb/s links between Boca and Jupiter Campus

• FAU’s high-performance computing cluster, named KoKo, consists of 7,120 cores, and 87 GPU Accelerators (V100, A100, P100, K80)

• Full membership in the Sunshine State Education & Research Computing Alliance (SSERCA) since February 2014

• Full membership in InCommon since Spring 2012

• 820 TB parallel Beegfs storage

Services and Expertise

Florida Atlantic University is committed to providing and continually refining well-defined computational and data storage options, technical expertise, and support services. Through its unified cluster strategy for compute, big data, and storage, FAU provides diverse resources to the various University departments, supporting numerous research domains across the University.

Collaborative Framework

Together with fellow members of SSERCA, FAU is building a framework that allows researchers at one school to avail themselves of the expertise and resources at other member institutions. This collaborative “compute and storage” framework has successfully been deployed across three universities and continues to expand to other SSERCA member institutions.

Background

In May 2012, FAU began investigating the possibility of building a Central Research Computing Department with the express goal of supporting faculty and student research. Supported by a team in the Enterprise Systems Group in OIT, the High Performance Computing (HPC) Group is charged with investigating faculty requirements and implementing systems designed to support faculty and students. The group’s activities and priorities are set by the High Performance Computing Faculty Advisory Committee, whose members meet monthly. The committee’s current goals include introducing or expanding computational science to a broader segment of FAU faculty, students, and researchers.

Goals for Campus Infrastructure

Information technology has the ability to empower scientific research and education. Recent innovations in computational science have accelerated the need for researchers and students to have access to quality computational environments such as, but not limited to, HPC, HTC, and big data. Employees entering the workforce in the next decade will need to process data in parallel and must know how to handle such data on high performance networks. Florida Atlantic University’s mission is to prepare students to utilize these new technologies, and to provide a safe, user-friendly environment in which to achieve this.

The FAU administration has supported the notion of central computational computing technologies and has sanctioned the installation of the KoKo HPC and OpenNebula. The installation of HTC and BIGDATA technologies as a component of the KoKo cluster will also make BIGDATA available in the future.

The foundation of a strong cyber environment is a group of coherent and well-managed resources. The HPC team at Florida Atlantic University has determined that these resources fall into the following categories:

• Compute capacity and capability

• Storage capacity protected from disaster and unauthorized access

• Network capacity for sharing data, both large and small

• Access to expertise, advice, and training for each resource

Infrastructure

The Office of Information Technology currently supports four data centers, located in Boca Raton, Jupiter, Tallahassee, and Davie. These data centers provide central support for University activities such as email, web, and other associated services. High Performance Computing has resources in two data centers. The private cloud and academic computing servers are located in the 1,900-square-foot data center on the Boca Raton campus. The KoKo HPC is located in a dedicated data center on the Jupiter campus. It is connected to the Jupiter network core at 10 Gb/s and has access to the Florida Lambda Rail (FLR) and other campuses using a 10 Gb/s network link.

Network Capacity

Florida Atlantic University is connected to the Internet via the FLR by way of a 10 Gb/s line at the Boca campus and a 10 Gb/s line at the Jupiter campus. The Fort Lauderdale and Davie campuses are linked together using a virtual cloud switch at 1 Gb/s. The HBOI and Jupiter campuses are linked to the Boca campus by Comcast by a 10 Gb/s line. The Office of Information Technology routinely investigates ways to improve the network bandwidth to the Internet and between campuses. The aim, and necessity, is to increase these to 100 Gb/s and 10 Gb/s, respectively, over the next 5 years because a number of projects cannot be competitively developed without addressing these limits.

It is crucial that FAU increase the capacity between the Jupiter and Boca campuses with 10 Gb/s networking and possibly an additional 10 Gb/s network link at the Jupiter campus for improved LAN activities. This would improve network failover and resiliency and allow the computational resources at the Jupiter campus to take advantage of improved network performance.

IPv6 Deployment

Florida Atlantic University is planning for campus-wide IPv6 deployment, with much of the required infrastructure already in place. In particular, FAU’s networking core was recently upgraded to allow OIT to proceed with testing and deployment phases. Deployment is scheduled to begin with new network subnets, including the proposed science DMZ. Based on testing and validation results within these subnets, the research computing team will proceed with broader deployment.

Adoption of BCP 38

Florida Atlantic University has confirmed its network architecture is not susceptible to the exploits detailed in BCP 38. Such exploits are blocked both within and at the edge of all networks by means of a firewall, implemented by the Department of Information Security, part of OIT. The firewall prevents spoofing of IP addresses. Such configurations are implemented at the Boca, Jupiter, and Harbor Branch campuses.

Compute and Storage Capacity

Many current and future projects at FAU require the storage of large amounts of data and compute capacity to process the data. Over the past few years, it has been increasingly challenging for individual researchers and departments to provide the required storage and computational resource research needs. The complexity of the software and hardware, and the cost of housing and ongoing maintenance, make a shared infrastructure approach more cost-effective for the University and researchers. By seeking support from the subject-matter experts in the OIT HPC Group, faculty members and departments are better able to focus their resources on research and academics.

The University is committed to continuing growth of the HPC to a total of 6,400 cores and over 1 PB of storage, while remaining at the current Jupiter data center. Pending availability of space and network infrastructure improvements, HPC computing hardware may then expand to the second Jupiter data center and/or to the Boca campus.

Integration with Statewide Infrastructure

Florida Atlantic University is working collaboratively with other Florida research institutions to build a framework for sharing data and compute cycles across the state. The University is an active member of the Sunshine State Education & Research Alliance (SSERCA), whose members include Florida State University, the University of Florida, the University of Central Florida, the University of Miami, and the University of South Florida. The alliance holds quarterly meetings and hosts a booth at the annual Super Computing Conference. SSERCA enables FAU to build on the knowledge of other Florida institutions and access a wide array of compute resources.

Implementation of XSEDE and InCommon

Florida Atlantic University acknowledges that to collaborate with other institutions, the University must offer a federated authentication system. Through its partnership with InCommon and FAU’s implementation of CAS and Shibboleth, the University is able to bridge its campus-wide authentication system with CiLogon, XSEDE, Globus, Research.gov, and many other third parties. The HPC Group is currently working to add compatibility with XSEDE resources and to implement assistive technologies that enable researchers to share data quickly with other sites.

Sustainability of Proposed Science DMZ

The proposed science DMZ is integral to FAU’s strategic vision for cyberinfrastructure expansion. Consequently, the infrastructure will be centrally managed and operated through OIT. After the 2-year award period, OIT commits to sustaining all network hardware and software associated with this award on an ongoing basis. The long-term cost of maintaining AT&T Wavelength service at 10Gb between the Boca Raton campus and Harbor Branch and Jupiter campuses is expected to be covered through external awards based on a shared-use, grant-funded model. Research networking fees will be incorporated as part of future grant applications, consistent with research computing fees at other institutions. From a pragmatic perspective, compliance is supported by the network’s hub-and-spoke design, allowing networking endpoints to be readily removed or swapped (e.g., from 10Gb/s to 1Gb/s). To allow sufficient time for this cost structure to become institutionalized, and to give investigators adequate time to adjust for these costs in their proposal submissions, OIT is committed to a de-escalating support schedule. In the first year beyond the award period, OIT commits to funding 75% of the AT&T service charges for the links connecting the Boca Raton campus to the Jupiter and Harbor Branch campuses. In the second year beyond the award period, OIT commits to funding 25% of these charges. These commitments are documented in the attached commitment letter from the Office of Information Technology.