fix: run continuous jobs immediately [DHIS2-15276] #15630
Conversation
|
Kudos, SonarCloud Quality Gate passed! |
Codecov Report
@@ Coverage Diff @@
## master #15630 +/- ##
============================================
- Coverage 66.19% 66.19% -0.01%
+ Complexity 31256 31253 -3
============================================
Files 3487 3487
Lines 129879 129887 +8
Branches 15166 15170 +4
============================================
- Hits 85979 85975 -4
- Misses 36814 36824 +10
- Partials 7086 7088 +2
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 3 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
| void runIfDue(JobConfiguration config); | ||
|
|
||
| /** | ||
| * Manually runs a job. OBS! This bypasses any actual checking if the job is due to run. When this |
There was a problem hiding this comment.
what does OBS stand for? seeing it in the code a good bit and have no idea.
There was a problem hiding this comment.
OBS! is how you say "oops" in Sweden and I think Norway as well. Here it is often used to catch the eye to say: pay attention. I thought it is more international but maybe not. Maybe use the English form? Oops?
There was a problem hiding this comment.
ahhh ok. Sounds like the English alternative to 'N.B.' (which is a Latin acronym 😄). Now that I know, it's fine.
The issue is that during tests when a test e.g. performs an import the actual import is just some milliseconds. But the first import would execute synchronous with the 20sec scheduler loop. So it is done shortly after the loop started to run. The queue would be empty again as the next test is waiting for the previous one to finish which requires finishing the import. So the next test runs again shortly after the loop cycle started, it then adds its import but at that point since the queue was empty the delay of almost 20 seconds has to be "payed" again essentially causing each test that has an import to take 20 seconds which causes overall execution time of the e2e tests go from 15min to 2h.
To prevent this test issue the continuous jobs are directly added to the queue and also will spawn a worker if no queue was present (and worked) at that time. This should remove the 20 second delay for this type of jobs. The cost is that now there is a concurrent part in the scheduler where both the scheduling loop thread and async web worker threads from
executeNowwill add to the queue and potentially spawn worker.As the accidental spawning of more than one worker is no issue this does not need to be synchronized the hard way.