This presentation was created using reveal.js, leveraging the org-reveal package in Emacs.
Type ‘?’ to see a list of the keyboard shortcuts.
- Swipe or use the arrow keys / n / space to navigate.
- Navigate/Swipe horizontally for main slides
- Navigate/Swipe downwards for sub-slides (if any)
- Menu key (bottom right corner) to jump to slides.
- Hit ‘o’ for an overview of the slides.
- Available distributed simulation solver licenses were drastically under-utilised due to the lack of a cohesive management system to run distributed simulations.
- Team workflow and productivity was severely impacted due manual firing of simulations.
- It was extremely cumbersome to submit test simulations to facilitate learning!
- Potentially: A huge change could be achieved with a simple system to fire simulations 24/7, utilizing the entire capacity through a logical and optimized basis.
- Reliable, automated and continuous software license utilisation in a First-In-First-Out (FIFO) basis, without manual intervention.
- Streamlined project management and resource utilisation, keeping track of multiple, distributed ANSYS CFX simulations of various types and priorities.
- Customised workflow and simplified interface to submit simulations for the entire team.
Hierarchy based logic: loop through a set order of folders and run the latest simulation files. The folders are meant to serve as multiple priority levels, and at times simulation size filters.
- For example, a Small (short) run’s priority would be higher than a non-priority, larger job at any time
- And then a Priority folder whose runs would run first no matter what.
- Increased software license utilisation by 50+%.
- Saved 40min (75%) in a critical design step.
- Eliminated repetitive manual script setups to submit multiple simulations.
- Significantly improved project management and output.
- Enabled the use of consistent solver and memory utilisation parameters and settings, allowing efficient deployment and reducing inefficiencies due to errors.
- Allowed optimal or perfect utilisation of available licensing scheme, resulting in a significant increase in team output and productivity.
- Written with Python 2.7, using portable python, Spyder, Notepad ++ and Sublime Text 3.
- System Scheduler
- Documentation and presentation re-prepared using Emacs and org-mode and org-reveal at a later time.
This code was originally developed for my R&D Center @ Wilo SE. A modified (and more generic) version of the same is available here. It is simply meant to fire away simulations, triggered every minute by a scheduler software. (Tested with the free version of *System Scheduler*).
This was achieved through a simple python script logic: loop through a set order of folders and run the latest simulation files. The folders are meant to serve as multiple priority levels, and at times simulation size filters.
- For example, a small (short) run’s priority would be higher than a non-priority, larger job at any time
- And then a Priority folder whose runs would run first no matter what.
The idea behind the project was to create a multi-platform job scheduler for ANSYS CFX that has a balance between sophistication and ease of deployment (and management), and of course customised to the team workflow.
Considering the numerous IT constraints and the situation at hand, this script just ticked all the boxes rather elegantly.
Open source solutions do exist. As of today, my approach would be different and I would leverage Pandas in python for a database of simulations and other automation tools like Airflow.
Job schedulers and load balancing programs are relatively sophisticated and complex to setup with several pre-requisites and constraints. Such complexity dictates expensive commercial support and licensing considerations for a team requiring a reliable system.
Once called, the program basically loops through pre designated folders and lists .def files based on the last modified date available in Windows.
The system interaction is via BASH scripts created via the Python code, as well as the python OS library. Using this approach, with flags to check the program status was just simpler than building some sort of constantly running python daemon.
There are several in-built flags to support setting priorities, pausing a particular cluster, logging data and troubleshooting.