Technische Universität Braunschweig
  • Study & Teaching
    • Beginning your Studies
      • Prospective Students
      • Degree Programmes
      • Application
      • Fit4TU
    • During your Studies
      • Freshmen-Hub
      • Term Dates
      • Information for Freshman
      • Practical Information
      • Additional Qualifications
      • Financing and Costs
      • Special Circumstances
      • Campus life
    • At the End of your Studies
      • Discontinuation and Credentials Certification
      • After graduation
      • Alumni
    • For Teaching Staff
      • Strategy, Offers and Information
      • Learning Management System Stud.IP
      • Team Teaching and Media Education
    • Contact
      • Student Advice Centre
      • Academic Advice Service
      • Admissions Office
  • Research
    • Research Profile
      • Core Research Areas
      • Clusters of Excellence
      • Research Projects
      • Research Centres
    • Early Stage Researchers
      • Promotion of early career scientists
      • PhD-Students
      • Postdocs
      • Junior research group leaders
      • Junior Professorship and Tenure-Track
      • Habilitation
      • Service Offers for Scientists
    • Research Data & Transparency
      • Transparency in Research
      • Research Data
      • Open Access Strategy
      • Digital Research Announcement
    • Research Funding
      • Research funding
    • Contact
      • Research Services
      • Academy for Graduates
  • International
    • International Students
      • Why Braunschweig?
      • Degree seeking students
      • Exchange Studies
      • Doctorate (PhD)
      • Refugee Students
      • Welcome Programme
      • TU Braunschweig Summer School
    • Scientists
      • Mobile Researchers at the TU Braunschweig
      • Research Services and European Office
    • Language and intercultural competence training
      • Learning German
      • Intercultural Communication
    • International Profile
      • Internationalisation
      • International Cooperation
    • International House
      • Information for first semester students
      • Contact
      • News and Events
      • Advisory Services
      • Location
      • About us
  • TU Braunschweig
    • Our Profile
      • Aims & Values
      • Regulations and Guidelines
      • Alliances & Partners
      • Facts & Figures
      • Our History
    • Career
      • Working at TU Braunschweig
      • Vacancies
    • Economy & Business
      • Knowledge and Technology Transfer
      • Entrepreneurship
    • General Public
      • Access to the University Library
    • Media Services
      • Communications and Press Service
      • Communications and Press Service
      • Film and photo permits
      • Advices for scientists
      • Topics and stories
    • Contact
      • General Contact
      • Getting here
  • Organisation
    • Presidency & Administration
      • Presidency
      • Designated Offices
      • Administration
      • Committees
    • Faculties
      • Carl-Friedrich-Gauß-Fakultät
      • Faculty of Life Sciences
      • Architecture, Civil Engineering and Environmental Sciences
      • Faculty of Mechanical Engineering
      • Fakultät für Elektrotechnik, Informationstechnik, Physik
      • Faculty of Humanities and Studies in Education
    • Institutes
      • Institutes from A to Z
    • Facilities
      • University Library
      • Gauß-IT-Zentrum
      • International House
      • Sports Centre
      • Facilities from A to Z
    • Equal Opportunity Office
      • Equal Opportunity Office
      • Family
      • Diversity for Students
  • Search
  • Quicklinks
    • People Search
    • Webmail
    • Campus map
    • CloudStorage
    • Messenger
    • Cafeteria
    • Courses
    • Stud.IP
    • Library Catalogue
    • IT Self-Service
    • Information Portal (employees)
    • Link Collection
    • DE
    • EN
    • IBR Twitter
    • IBR YouTube
    • Facebook
    • Twitter
    • Instagram
    • YouTube
    • LinkedIn
Menu
  • Organisation
  • Faculties
  • Carl-Friedrich-Gauß-Fakultät
  • Institutes
  • Institute of Operating Systems and Computer Networks
  • IBR Knowledge Base
Logo IBR
IBR Login
  • Institute of Operating Systems and Computer Networks
    • News
    • About us
      • Whole Team
      • Directions
      • Floor Plan
      • Projects
      • Publications
      • Software
      • News Archive
    • Connected and Mobile Systems
      • Team
      • Courses
      • Theses
      • Projects
      • Publications
      • Software
      • Datasets
    • Reliable System Software
      • Team
      • Advent(2)
    • Algorithms
      • Team
      • Courses
      • Theses
      • Projects
      • Publications
    • Microprocessor Lab
    • Education
      • Winter 2023/2024
      • Summer 2023
      • Theses
    • Services
      • Library
      • Mailinglists
      • Webmail
      • Knowledge Base
      • Wiki
      • Account Management
      • Services Status
    • Spin-Offs
      • Docoloc
      • AIPARK
      • Confidential Technologies
    • Research Cooperations
      • IST.hub

Running "Heavy" Jobs on IBR Servers

AuthorFrank Steinberg
KeywordsSimulations Simulationen Batch Jobs Compute
CategoriesAdministrivia, Software

Available Hosts

"Heavy Jobs" like ns-2 simulations that you might want to run on IBR hosts need lots of resources of mainly three types: (1) CPU power for short runtimes, (2) system memory during execution, and (3) disk space for results (and maybe for temporary data).

Please choose parameters and hosts for your jobs very carefully before you run your jobs. When your jobs run out of system memory and the system starts swapping or thrashing, it does not help you and it bothers other users and administrators.

You can use tools like top(1), free(1), df(1) to find out how many CPUs a host has, how much system memory is installed and available, and where huge local temporary disk space (typically /opt/tmp) is available.

Here is some host specific information as of 2010-12, but it may change over time: predator, radiator, operator, animator, bierator have 64bit Linux operation systems and huge amounts of memory (6-16GB). Of course, you have to compile your programs for this environment, to make use of the large address space. Unimator should not be used because this host is usually used for short-term interactive sessions, for which people expect a reasonable responsiveness.

You might also use workstations from the linux pool.

How To Run Your Jobs

Before you start your jobs, please make sure where it will dump its results. It is not wise to write gigabytes of data via NFS to a remote filesystem like your home directory. Use one of the above mentioned local "tmp" directories, whenever possible. Please also try to estimake how much data will be written to output files and check the available disk space in advance.

Please, use nice(1) (or renice(1), if the process is already running) to run your heavy jobs! Always! This helps the operating system to schedule processes so that CPU intensive jobs don't bother interactive jobs too much and the system keeps relatively responsive to people working interactively on the host. Since interactive jobs generally require very few CPU cycles compared to your heavy jobs, this slows down your jobs only very very slightly (by ~1%), but it helps your fellows significantly!

On multi-processor hosts, you might decide to run as many processes in parallel as CPU cores are present (or maybe even one or two more, if there is some significant amount of I/O on your jobs). But still keep aware of the system's total resource limits!

You might want to hack some scripting around the command lines to start your jobs. This is not the place for a Script-Howto, but you might want to read the nohup(1) manual page and info page to find out how to start your jobs in the background, redirecting output to a file and not terminating when you logout.

In some rare cases, you might want to pause/resume jobs. You can do this by sending it STOP/CONT signals, see kill(1).

If you recognize that the system starts swapping (top(1), vmstat(8) can help to recognize this), please consider terminating your job! A system that starts swapping is slowing down significantly, so that in most cases you get faster results, when you restart your job from scratch with a smaller footprint or on a larger machine.

When your jobs are finished, please clean up the temporary disk space as early as possible! If you expect to need the immediate output of your jobs for a longer time for further processing, please try to find a way to condense it down to the relevant parts and/or to compress it.


last changed 2010-12-06, 16:28 by Frank Steinberg

For All Visitors

Vacancies of TU Braunschweig
Career Service' Job Exchange 
Merchandising

For Students

Term Dates
Courses
Degree Programmes
Information for Freshman
TUCard

Internal Tools

Glossary (GER-EN)
Change your Personal Data

Contact

Technische Universität Braunschweig
Universitätsplatz 2
38106 Braunschweig

P. O. Box: 38092 Braunschweig
GERMANY

Phone: +49 (0) 531 391-0

Getting here

© Technische Universität Braunschweig
Imprint Privacy Accessibility