Server Usage Policy

The SSCC has many powerful servers, but we have many more users. The servers are a shared resource, so it's essential that all users share them properly.

We therefore ask all our members to adhere to the following:

  1. Do not use more than 128 gigabytes of RAM on Winstat or WinSilo, including all the programs you are running. Instructions for tracking RAM usage are given below. Using more RAM can make the server unusable for everyone. (Linstat and LinSilo automatically limit each user to 350GB of RAM. Trying to use more RAM than is actually available will not go well on Linstat or LinSilo either, but the server will generally recover by killing jobs until it has memory available again.)
  2. Do not try to use more cores than a server actually has, either by running a single job that uses many cores or by running many jobs (see Computing Resources at the SSCC for server specifications). Software-specific instructions for tracking and setting the number of cores used are given below. Trying to use more cores than the server has will force the cores to switch between tasks repeatedly, slowing down the server for everyone.
  3. Do not put than 15 jobs in the Condor or CondorSilo queues at a time. Doing so will prevent others from using Condor/CondorSilo at all until your jobs are finished.

Exceptions to these policies may be made under rare circumstances. Contact the Help Desk if you believe your case merits an exception.

SSCC staff regularly monitor the load on each of the servers. If we notice jobs that are in violation of these policies, we will contact the job owner, but let the job finish as long as it is not causing problems for other users. However, if the job disrupts usage for others, it may be terminated without prior notice.

This will not keep the servers from getting busy on occasion, but it does ensure that they will not be overloaded by a single user or very small group of users.

Tracking RAM Usage

Most statistical software loads the data it is working on into RAM, and the amount of RAM used will be slightly larger than the size of the data used. Be aware of the size of your data set. If your data set takes a long time to load consider that a warning sign that you need to start paying attention to RAM.

Some programs will tell you how much RAM they are using. For example, Stata lists current memory usage among the properties of the data set.

To see how much memory any program is using on Winstat or WinSilo, right click on the gray bar at the bottom of the screen, choose Task Manager, go to the Processes tab, identify your job, and check the Memory column. To see how much memory any program is using on Linstat or LinSilo, run top.

Controlling the Cores Used by Popular Programs at the SSCC

Matlab

If you are doing parallel processing, run parpool(N) where N should be replaced by the number of cores you want to use. Alternatively, click the arrow by Parallel on the Home tab, then Parallel Preferences, and set Preferred number of workers in a parallel pool to N.

MPlus

Include the following line in your program:

analysis: process = N;

where N should be replaced by the number of cores you want to use.

Stata

The number of cores used by Stata depends on which version you run. The default version on Winstat and WinSilo is Stata/SE, which uses one core, but you can use 16 cores if you run Stata/MP. On Linstat and LinSilo the default is Stata/MP, which will use up to 32 cores. Running Stata/MP at the SSCC has details.

R

R will only use one core unless you are using a package designed for parallel processing. If you are using such a package, consult its documentation to see how to control the number of cores it uses. For example, if you are using parLapply you should run makeCluster(N)where N should be replaced by the number of cores you want to use..

SAS

You do not need to set the number of cores used by SAS. SAS is set to use 4 cores on Winstat and WinSilo and 16 cores on Linstat and LinSilo.

SPSS

You do not need to set the number of cores used by SPSS. It will use 4 cores on Winstat and WinSilo.

User-written programs

If you are using OpenMP with FORTRAN, you can set the number of cores to use with:

!$ call omp_set_num_threads(N)

where N should be replaced by the number of cores you want to use. If you are using MPI such that your job can take advantage of multiple machines, consider asking for an account on the SSCC's High Performance Computing Cluster.