SoFiA Production Update 6

Message boards : News : SoFiA Production Update 6

To post messages, you must log in.

AuthorMessage
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 239
Credit: 7,636
RAC: 0
Message 1143 - Posted: 17 Jan 2018, 7:01:25 UTC

Hello everyone,

For this week, I'm planning on running a small set of 100MB workunits through SoFiA, just as a general test. There's only 705 workunits, but each workunit should take substantially longer to process than the ones we've been doing so far. For these test workunits, we're running all ~400 parameters on each workunit, rather than splitting the parameters up in to different runs. This is only going to be the case for the test workunits, the production ones will use split parameter sets just like the 10MB workunits did.

The SoFiA VM has been updated to use the latest version of SoFiA. Hopefully, this will help increase general stability and decrease some of the problems we've been encountering.

Each VM instance will now use up to 3072MB of memory, a huge increase from the previous 256MB. I'm sorry about this, but SoFiA doesn't seem to have particularly optimal memory usage, and my local tests were running into out of memory problems at anything less than 3GB of memory. You may want to limit the number of SoFiA instances your machines run if you begin to get performance issues. I've also reported the rather large memory usage to the developers, as it may be something they can improve on in future versions.

I'll be trying to visit the forums daily to keep an eye on any issues that might crop up, so hopefully I'll be able to resolve small problems in my spare time instead of having to postpone everything until next Wednesday.

I hope this goes smoothly.

Sam.
ID: 1143 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
LumenDan

Send message
Joined: 9 Feb 17
Posts: 101
Credit: 182,042
RAC: 518
Message 1144 - Posted: 17 Jan 2018, 9:02:52 UTC - in response to Message 1143.  

Each VM instance will now use up to 3072MB of memory

It might pay to give the 100MB SoFiA application a different name to the 10MB version even if there is no difference in the software (at least in the production run).
With two separate application names you could adjust the plan class for the memory hungry application preventing units being sent to computers that won't cope.

You could also allow application opt-in via the project settings page so that contributors can chose to run the program only on computers that are assigned to a specific "location" (using the default school, work, home choices). Selecting applications by "location" won't matter while we only have one application running at a time but it would be handy if you do end up running applications in tandem once the real data comes in.

I'm looking forward to testing these new units.
ID: 1144 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
boddie

Send message
Joined: 8 Jan 18
Posts: 1
Credit: 0
RAC: 0
Message 1145 - Posted: 17 Jan 2018, 11:15:45 UTC - in response to Message 1144.  

my "Duchamp" project workunits aren't completing :(

BOINC downloaded about a dozen work units. Each estimated to take around 58 minutes.

When I noticed a number had been running for a quite a few hours I suspended the majority of them.

I have one that took HOURS to go from 99.999% to 100.000%. the status of that one for the last few hours (@100%) has been "running"! Elapsed for this one is over 22 hours.

I resumed others, one at-a-time, and now have 6 at 100.00%, and have suspended those waiting for the running one to progress past "running".
ID: 1145 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Seb

Send message
Joined: 12 Jan 18
Posts: 8
Credit: 18,140
RAC: 775
Message 1146 - Posted: 17 Jan 2018, 22:43:48 UTC

Works great for me, got one on my i5-6600, took about 30 minutes to complete.

Looking forward more WUs!
ID: 1146 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Bill F

Send message
Joined: 9 Feb 17
Posts: 31
Credit: 63,875
RAC: 17
Message 1147 - Posted: 19 Jan 2018, 2:27:38 UTC

I have an older WU that I suspect is a resend from a prior batch ... 50% failure rate on systems that have attempted it including mine.

Workunit 254676

Ran on Project Computer 80

AuthenticAMD AMD Phenom(tm) II X6 1045T Processor [Family 16 Model 10 Stepping 0]
Operating System Microsoft Windows 7 Ultimate x64 Edition, Service Pack 1, (06.01.7601.00)
BOINC version 7.8.3
Memory 8156.23 MB
Cache 512 KB
Swap space 16310.62 MB
Total disk space 931.41 GB
Free Disk Space 601.74 GB

Thanks
Bill F
Dallas TX
ID: 1147 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Bill F

Send message
Joined: 9 Feb 17
Posts: 31
Credit: 63,875
RAC: 17
Message 1148 - Posted: 19 Jan 2018, 4:42:06 UTC - in response to Message 1145.  
Last modified: 19 Jan 2018, 4:43:29 UTC

Boddie

Is your V-Box reasonably current and functioning on other projects ?

current version is 5.1.26

Bill F
Dallas TX
ID: 1148 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Yavanius
Volunteer moderator
Avatar

Send message
Joined: 12 Feb 17
Posts: 146
Credit: 169,205
RAC: 98
Message 1149 - Posted: 21 Jan 2018, 22:39:34 UTC - in response to Message 1148.  


Is your V-Box reasonably current and functioning on other projects ?

current version is 5.1.26


I think you meant 5.2.26 unless you're stuck in a temporal flux. :) Of the 5.1 series; 5.1.32 was the last one.

~Y
ID: 1149 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Bill F

Send message
Joined: 9 Feb 17
Posts: 31
Credit: 63,875
RAC: 17
Message 1150 - Posted: 22 Jan 2018, 7:36:04 UTC - in response to Message 1149.  

I was giving the Standard version that you get with BOINC and keeping in mind your posting that version 5.2 does not work with SoFiA

B
ID: 1150 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Seb

Send message
Joined: 12 Jan 18
Posts: 8
Credit: 18,140
RAC: 775
Message 1151 - Posted: 22 Jan 2018, 11:28:48 UTC - in response to Message 1150.  

It does work, pretty well even.
ID: 1151 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Crystal Pellet

Send message
Joined: 10 Feb 17
Posts: 19
Credit: 41,882
RAC: 19
Message 1179 - Posted: 29 Jan 2018, 13:06:38 UTC - in response to Message 1143.  
Last modified: 29 Jan 2018, 13:07:15 UTC

...
For this week, I'm planning on running a small set of 100MB workunits through SoFiA, just as a general test. There's only 705 workunits, but each workunit should take substantially longer to process than the ones we've been doing so far. For these test workunits, we're running all ~400 parameters on each workunit, rather than splitting the parameters up in to different runs. This is only going to be the case for the test workunits, the production ones will use split parameter sets just like the 10MB workunits did.

The SoFiA VM has been updated to use the latest version of SoFiA. Hopefully, this will help increase general stability and decrease some of the problems we've been encountering.

Each VM instance will now use up to 3072MB of memory, a huge increase from the previous 256MB.
.
.
.
I hope this goes smoothly.

Sam.

Those tests aren't going smoothy.
Most of them are returned as an error, cause they extended the time limit.
On my machine they should be ready within 7.5 hours, but after 2.5 hours run time they have done 47 parameters out of the 486 to do even when I extended the memory from 2048 to 3072 again.
ID: 1179 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 239
Credit: 7,636
RAC: 0
Message 1188 - Posted: 31 Jan 2018, 1:18:09 UTC - in response to Message 1179.  

Alright I'm going to look into chopping the parameter set significantly again, because these runtimes are ridiculous.
Sorry about this.
ID: 1188 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Yavanius
Volunteer moderator
Avatar

Send message
Joined: 12 Feb 17
Posts: 146
Credit: 169,205
RAC: 98
Message 1192 - Posted: 31 Jan 2018, 2:28:11 UTC - in response to Message 1150.  

I was giving the Standard version that you get with BOINC and keeping in mind your posting that version 5.2 does not work with SoFiA

B


The folks at Berkeley are going to try to keep the VirtualBox version packaged updated. There are changes happening internally to address these kind of issues in the future. It's a good rule of thumb to wait in software updates one version for a minor update and a couple for a major version updated (e.g. 5.1 -> 5.2) in case things break (read the change logs some time when you go to get a new version of VB!) unless there is something you specifically need in the new update or you are specifically instructed by a project.

When 5.2 was released by Oracle it broke all the projects that had VB clients. Sam released his own Wrapper for 5.2 series due to the delay from BOINC in getting a new wrapper package out. I have not seen mention of a new official Wrapper for BOINC (although I may have missed it) and have no idea currently if the other projects with VB clients have received anything or made any kind of work-arounds of their own. I can only say SourceFinder will work with 5.2.x. I recommend you check the forums at the other projects for updates if you want to use 5.2. I have not seen any news items about it.

If you're running other BOINC projects and utilizing their VirtualBox clients, we recommend 5.1.32 which the majority of folks out in the BOINC community reported as stable. If not, sofar since Sam updated the Wrappers, we haven't an issue with Virtualbox 5.2.x (not to be confused with the actual Sofia client issues) and you can update as you desire. :)

~Yav
ID: 1192 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : News : SoFiA Production Update 6


©2018 ICRAR