Managing types of running tasks
17 Feb 2025 20:33:34 UTC
Topic 232106
(moderation:
On my set up I get the best throughput when concurrently running a single "All-Sky Gravitational Wave search on O3" task and a single "Binary Radio Pulsar Search (MeerKAT)" task. If I run a single gravitational wave task then my GPU is not fully utilised, likewise if I run a single MeerKAT it isn't either. This is proven by the fact that the run times do not increase significantly when running the two types of task concurrently.
However I do not have enough GRAM to run two gravitational wave tasks and for some reason when multiple MeerKAT tasks ever start running then it crashes, BOINC then proceeds to try running as many of them as possible and I got lots of errors and invalids, which helps no one. This happened again last night despite there still being enough older gravitational wave tasks to run, BOINC decided it was better to run multiple newer MeerKAT tasks at the same time.
Therefore I would like to be able to configure my system to run precisely one gravitational wave and one MeerKAT task. Since the GPU utilization factor for each task does not seem to be able to rule out trying to run two of the same type of task, I tried setting up a app_config.xml but then I didn't get any work, although I might have done something wrong. Does anyone have any tips?
Mr Anderson wrote:... If I
)
Your GPU shows as a 4GB AMD RX570. I have a lot of those as well, but running under Linux and not Windows. I run both types of tasks, but never on the one machine because I seem to recall seeing other users mentioning that the two apps tend to interfere with each other.
Your results seem to indicate that as well since they show huge variations in run time, contrary to what the above quote seems to suggest. For example, you currently have 21 validated BRP7 with run times ranging from a low of 1754 secs up to a high of 2253 secs. On one of my machines, single BRP7 tasks consistently run just over 1500 secs with very little variation. If I run 2 concurrent BRP7 tasks, the run time effectively doubles, indicating that a single BRP7 task does consume the available GPU resources, contrary to your assertion.
For my GPUs running O3AS, a single task took around 5500 secs, when I first tested. All my 4GB RX570s can run 2x (two concurrent O3AS tasks) and have done so right from the beginning. When running O3AS tasks 2x, the actual run time is around 8200 secs, with little variation, giving an effective task time of around 4100 secs.
You have 9 valid O3AS tasks currently, with run times from 13,277 secs up to 20,374 secs. I don't see how you can reconcile this with your quote above. My guess is that the extremely slow and variable times for O3AS tasks do indicate that the two apps are fighting with each other for resources.
Have you actually suspended BRP7 tasks and tried running O3AS tasks on their own? Have you tried running two concurrent O3AS to see the further improvement? Maybe Windows might not let you run O3AS tasks 2x on a 4GB GPU. There's no problem with doing this on Linux.
Cheers,
Gary.
Mr Anderson
)
Gary Roberts is correct when he says "I run both types of tasks, but never on the one machine because I seem to recall seeing other users mentioning that the two apps tend to interfere with each other." And I know that Gary runs Linux for all of his computers, as do I. I can't say for sure how Windows would handle your programs, but it's more of a personal choice.
I recommend choosing one or the other, but not both BOINC Einstein programs. The O3AS and BRP7 do want to fight each other for dominance when they run on the same GPU. You mentioned that when you made an app_config.xml file that you didn't receive any work units. Could you post your app_config.xml file? And do you have an app_info.xml file? If so, could you post it also? It would be helpful for us if you could.
Proud member of the Old Farts Association
Thanks for your analysis
)
Thanks for your analysis Gary. I have tried O3AS and BRP7 running by themselves, I usually run single O3AS tasks because I am more interested in that project and because it is less fuss. Only when I'm feeling lucky do I bring in some BRP7 tasks to run concurrently along with some previously downloaded O3AS tasks (and try to ensure that they will all run out before the older O3AS tasks do). From my analysis I get the following when running BRP7 and O3AS one at a time:
BRP7: these are indeed quite consistent and are around 1780 secs => about 162000 credits per day
O3AS: these can vary greatly e.g. 13277 - 20374 but typically are around 15000 secs => 115200 credits per day
(The 20374 second task was running by itself, no other concurrent tasks. Perhaps it was fighting for resources against Windows because Windows had something really "important" to do.)
Running both concurrently does make both take longer but not twice as long:
BRP7: about 2150 secs => about 134000 credits per day
O3AS: about 18500 secs => about 93400 credits per day
Together that makes about 227400 credits per day. If I were to only run BRP7 that would be 29% less and if I were to only run O3AS then that would be 49% less, which is significant.
I have not tried running two of the latest type of O3AS concurrently as I don't see how there could be enough GRAM given Windows' propensity for running lots of meaningless things that use it (even the calculator uses 20 MB). Currently with only one O3AS running, 2.3 out of 4.0 GB is used. Last year at times I did run two of the older O3AS app concurrently but to free up GRAM I had to kill the explorer process and that leaves the screen blank except for any running windows (no desktop and start menu) which is a bit frightening for other users.
@GWGeorge007, this was the
)
@GWGeorge007, this was the app_config.xml file that I tried out in December (after the new O3AS tasks were introduced). As mentioned, it didn't seem to work but I will not be at all surprised if it is incorrect. I did not know about the app_info.xml files so I guess that says something...
<app_config>
<app>
<name>einsteinbinary_BRP7</name>
<max_concurrent>1</max_concurrent>
<gpu_versions>
<gpu_usage>0.4</gpu_usage>
<cpu_usage>0.1</cpu_usage>
</gpu_versions>
</app>
<app>
<name>einstein_O3AS</name>
<max_concurrent>1</max_concurrent>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>0.1</cpu_usage>
</gpu_versions>
</app>
</app_config>
While your AMD Radeon RX 570
)
While your AMD Radeon RX 570 GPU does have 4096MB for VRAM, I'm not sure if you can run two tasks of O3AS on it. I run a 10GB VRAM RTX GeForce 3080 with three O3AS tasks because they take up to 2238 MiB for each task. I'm just sayin'...
I did eliminate your <max_concurrent>1</max_concurrent>.
Try either one of these app_config.xml files, but not both.
BRP7
<app_config>
<app>
<name>einsteinbinary_BRP7</name>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>0.1</cpu_usage>
</gpu_versions>
</app>
</app_config>
O3AS
<app_config>
<app>
<name>einstein_O3AS</name>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>0.1</cpu_usage>
</gpu_versions>
</app>
</app_config>
If you find that your RX570 won't run two tasks, or they take a very long time to complete, change your gpu_usage to 1.0.
To identify how well your GPU is doing in the tasks, run a baseline of one task and see what times you get. I would suggest running it for 24 hrs, then add them all up and find the average.
If you can run 2 tasks of either BRP7 or O3AS, then run either one (but not both) for at least 24 hrs, find your average times for running 2 tasks and then divide by 2. If your GPU runs 2 tasks effectively in less time than your first baseline, then your better off running 2 tasks.
Proud member of the Old Farts Association
I must be missing something
)
I must be missing something as I don't seem to understand the problem here. Why would BRP7 and O3AS interfere with each other since they don't run concurrently? If they do run concurrently, how does them running concurrently happen? I usually have both of those projects enabled and get a mix of both tasks and they never run at the same time. BOINC decides how many BRP7s to run in a row before switching to running some O3AS tasks. Never a BRP7 and an O3AS running at exactly the same time. I have a Titan V on Windows 10 but I don't see why that should matter.
AndreyOR wrote:I must be
)
Depending on GPU capabilities, you can have multiple GPU tasks running concurrently and this often better utilizes the GPU. See the "GPU utilization factor" settings in Project Preferences. For example if you set "GPU utilization factor of BRP apps" to 0.5 then you can have two BRP7 tasks running at the same time. The tasks will often take a bit longer but if it's less than twice as long, which in my experience is usually the case, then you're getting more work done than if they were executing one after another. I'm pretty sure your Titan V will enjoy having a bit more to chew on.
In my case I cannot run two O3AS tasks concurrently because I don't have the GRAM for it. I cannot run two BRP7 tasks concurrently because it crashes. But I can run an O3AS and a BRP7 concurrently and, yes, I have averaged out and reviewed the numbers and found that combination to produce more credit over a given time period.
Mr Anderson wrote: ... But I
)
That's the part that I'm unclear about. How does this happen? Why would BOINC start 2 different project's tasks on the same device (GPU) at the same time?
What is your setting for "Switch between tasks..."? If it's really small, that's the only possible way I can think of but even that would be switching between tasks, not running them concurrently.
Just try it out, your
)
Just try it out, your graphics card is certainly up to it. Go into your project preferences in the web interface (not BOINC) and set "GPU utilization factor of GW apps" to 0.5 and submit. When you get new GW tasks then BOINC will start running two of them at the same time (not switching from one to another but running them concurrently). Then just observe how long they take to run.
Currently they seem to take about 2200 seconds on your machine so when running two of them concurrently they might take a bit longer, say 3000 seconds, but remember that you're getting 2 tasks done in those 3000 seconds instead of 2200+2200=4400 seconds if running them sequentially. This happens because a single task often cannot fully utilise the GPU so a second (or third) task can use unused resources and processing power.
With 12 GB you could even try 0.3 for three concurrent tasks or maybe even 0.25 for four, however of course the returns will diminish. If it doesn't run well and/or your system gets unstable then you can change the setting back to 1.0 and when you get new work then BOINC will go back to running one at a time.
I think it's important to
)
I think it's important to note here that when upping your GPU usage from 1x to 2x or 3x, you should also take note of the GPU Utilization factor which can be easily found by using the terminal in Linux with "nvdia-smi" (text without quotes), or "watch -n 1.0 nvidia-smi" like this:
(NOTE: This command is available in Linux, which I am running. I do not know if it, or something like it is available in Windows. The 'watch' command is a 'live' variation of the command "nvidia-smi")
When looking at nvidia-smi you'll notice that I have 2 GPUs, a 3090 and a 3080, and they are both running 3 consecutive tasks of O3AS. Also, the utilization factor is at 100% for both, meaning that neither one can effectively produce any more than 3 tasks even though they are using much less memory (for the GPU) than what is actually available.
If I were to actually run them at 4x or even 5x tasks, of course the GPU-Util will remain at 100%, and my tasks will continue to run but the completion times for each task will take a direct hit and become worse (longer) than the 3x task time.
Just thought I'd toss in my $0.02 worth...
Proud member of the Old Farts Association