SoFiA Beta Update 3

Message boards : News : SoFiA Beta Update 3

To post messages, you must log in.

AuthorMessage
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 167
Credit: 7,636
RAC: 1
Message 775 - Posted: 20 Sep 2017, 1:26:26 UTC
Last modified: 20 Sep 2017, 2:54:12 UTC

Fixed a few bugs in the assimilator.
Improved logging in the validator.

Out of the 1736 work units sent out in the last update, I have the results for 1723 of them stored in our cloud storage system. That's a 99% success rating!

There were a couple of issues with the assimilator that were preventing the results from being properly stored in our database though. We store sources found in our database as well as storing the raw files returned by a client in cloud storage.

Hopefully this will be the last week of the SoFiA beta, and then we can proceed in to full processing mode!

Edit: I've also updated the project description that appears on the main page to include SoFiA.
ID: 775 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Dave Peachey

Send message
Joined: 9 Feb 17
Posts: 46
Credit: 767,358
RAC: 30
Message 779 - Posted: 20 Sep 2017, 16:01:34 UTC - in response to Message 775.  
Last modified: 20 Sep 2017, 16:01:48 UTC

Sam,

Other than being out of work already, that seems to have gone off without a hitch; the short WU run times (8 mins per WU on my machine) is certainly getting the job done!

However, I wonder whether there's scope to increase the number of parameters when you generate the next batch of test or the eventual "live" WUs (your post of 5th Sept indicated you had reduced the number of parameters being run by the SoFiA application against those being run by the Duchamp application) in order to give us all a bit more meat to chew on.

Also, one other thing re housekeeping ... it seems as though historic parameters_sofia_n.tar.gz files are sticking on my machine and not being cleaned out when the updated set of parameters/WUs are being delivered and processed. Whilst this isn't much of an issue given the size of those files, it might be worth seeing what can be done to improve the application housekeeping processes.

Dave
ID: 779 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Chris Skull

Send message
Joined: 10 Feb 17
Posts: 26
Credit: 63,909
RAC: 35
Message 780 - Posted: 20 Sep 2017, 17:13:09 UTC

some units never end... 100% progress, 1% CPU usage... i cancel them after more than 2h runtime.
Greetz
Chris
ID: 780 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 167
Credit: 7,636
RAC: 1
Message 783 - Posted: 27 Sep 2017, 2:25:13 UTC

Once SoFiA has moved out of beta, I can push out a "file delete" command to remove all of the old parameter files on everyone's machines.

The never ending work units have been an issue here from the start.. The beta work units in this batch should be extremely quick, taking only 8 or so mins as Dave said.
Do the units complete and upload if you restart BOINC?
ID: 783 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : News : SoFiA Beta Update 3


©2017 ICRAR