Facilities

BH Lab — The three CCRG clusters are housed in a large computer room (74-2060), know as the Black-Hole Lab", which is visible behind a large glass wall. The lab showcases the center's research and its commitment to green computing.

Introduction

New Horizons (500 cores)is our oldest cluster. It is a hybrid cluster with 500 computational cores purchased via two NSF grant grants PHY-0722703, DMS-0820923.

Our research focus is on some of the most extreme phenomena in the universe, where the strongest gravitational and magnetic fields interact with ultra-relativistic matter and high-energy radiation, that can only be studied through advanced, large-scale computation and visualization.

Blue Sky (1040 cores) is a hybrid cluster consisting of 50 Dual socket Intel Sandy Bridge Xeon E5-25630 nodes with 16 cores and 64GB RAM per node, as welll as 20 dual socket Intel Nehalem Xeon X5690 nodes with 12 cores and 48GB of RAM per node. All nodes are connected via a QDR Infiniband fabric, and have direct access to 261TB Lustre filesystem. Two of the Sandy Bridge nodes als contain dual Intel Phi coprecessors. It was purchased through an NSF grants: PHY-1229173, A ST-1028087.

Therefore, computer clusters are a key element to the success of our research programs. The clusters are also used as testbed for developing novel computational techniques and software, which also form part of the research mission of the CCRG.

All three clusters are designed for high-speed, large memory, and fast interconnect, and are optimized for the maximum performance of our simulation software.

Green Prairies (1296 cores) is a new state-of-the art computational cluster to support research at the frontiers of gravitational physics and relativistic astrophysics. The system consist of a 40 nodes, each with dual Intel Skylake 6130 16-core CPUs, and interconnected with an Intel Omnipath fabric. 38 nodes have 192GB of RAM, while two nodes have 384GB of RAM. All nodes are connected to a high-speed 768TB Lustre filesystem. It was purchased via an NSF MRI grant PHY-1726215.

All clusters and storage system were produced by Aeon Computing, and are cooled using a an in-rack green cooling solution provided by a local company, OptiCool Technologies.

Storage The central research computing facilities at RIT makes available a new Cold Storage system with a total of 2 PB available for our use. Each node consists of 2x Intel Scalable 4108 (16 cores) processors, 96GB Memory, 12x 12TB HDDs (144TB RAW), 2x 500GB NVMe storage 25Gb Network Interface. The total raw storage is 3.4PB, and around 2 PB usable with erasure coding.

In addition to local cluster facilities, scientists at the center also use national supercomputing resources such as the: NSF-funded petascale Blue Waters systems at the Illinois’ National Center for Supercomputer Applications (NCSA), available through allocations grants (ACI-0832606, ACI-1516125, PHY-0722703), NSF-funded XSEDE resources, and has recently been awarded allocation in the NSF-funded Petascale computing system, Frontera, was awarded for deployment at the Texas Advanced Computing Center (TACC).

More information about our allocation awards see the following pages: RIT's NCSA's Blue Waters Allocation, TACC's early science allocation, also the NSF's Cyberinfrastructure and Major Facilities Presentation for an overview on these large allocations and systems.