SoFiA Beta Update 4

Message boards : News : SoFiA Beta Update 4

To post messages, you must log in.

AuthorMessage
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 167
Credit: 7,636
RAC: 1
Message 785 - Posted: 27 Sep 2017, 7:17:38 UTC

Fix yet another bug in the assimilator.

Modified the way SoFiA returns output files. There was an issue with various parameter combinations causing SoFiA to return outputs in different formats. This should be accounted for properly now.

Built a new, extended parameter set for SoFiA testing. We're now testing about 100 different combinations.

Pushed out new work units using the extended parameter set.
ID: 785 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Dave Peachey

Send message
Joined: 9 Feb 17
Posts: 46
Credit: 767,358
RAC: 30
Message 787 - Posted: 27 Sep 2017, 7:47:35 UTC

Sam,

I'm getting a continuous stream of download errors for this new batch of work - the v7 parameter file is present and the individual WU files seem to download successfually but I have no supporting SofiaVM_1.03.vdi file in the ..\projects\.. folder.

Dave
ID: 787 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
LumenDan

Send message
Joined: 9 Feb 17
Posts: 73
Credit: 125,827
RAC: 188
Message 788 - Posted: 27 Sep 2017, 7:54:20 UTC - in response to Message 787.  
Last modified: 27 Sep 2017, 8:43:06 UTC

I'm also getting download errors for the virtual hard disk file:
27/09/2017 17:20:43 | duchamp | Giving up on download of SofiaVM_1.3.vdi: permanent HTTP error

Edit: I tried re-attaching to the project with the same result.
Are the Beta4 work units referencing the wrong Sofia VM version?
ID: 788 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Maurice Goulois

Send message
Joined: 17 Feb 17
Posts: 1
Credit: 18,194
RAC: 0
Message 789 - Posted: 27 Sep 2017, 8:45:11 UTC

Hi !

Download errors too on my side.
ID: 789 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sergey Kovalchuk

Send message
Joined: 9 Feb 17
Posts: 1
Credit: 4,283
RAC: 5
Message 790 - Posted: 27 Sep 2017, 9:19:23 UTC

You don't have permission to access /duchamp/download/SofiaVM_1.3.vdi on this server


so the file is present, but has the wrong attributes or owner
ID: 790 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
GCB

Send message
Joined: 9 Feb 17
Posts: 1
Credit: 61,704
RAC: 56
Message 791 - Posted: 27 Sep 2017, 13:10:59 UTC

Getting a similar issue here:

duchamp | Giving up on download of SofiaVM_1.3.vdi: permanent HTTP error

Log extract below

27/09/2017 14:05:33 | | [network_status] status: online
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: Connected to sourcefinder.theskynet.org (54.196.169.236) port 443 (#1098)
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: ALPN, offering http/1.1
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: successfully set certificate verify locations:
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: CAfile: C:\Program Files\BOINC\ca-bundle.crt
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: CApath: none
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (OUT), TLS header, Certificate Status (22):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (OUT), TLS handshake, Client hello (1):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: Connected to sourcefinder.theskynet.org (54.196.169.236) port 443 (#1097)
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: ALPN, offering http/1.1
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: successfully set certificate verify locations:
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: CAfile: C:\Program Files\BOINC\ca-bundle.crt
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: CApath: none
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (OUT), TLS header, Certificate Status (22):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (OUT), TLS handshake, Client hello (1):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (IN), TLS handshake, Server hello (2):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (IN), TLS handshake, Certificate (11):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (IN), TLS handshake, Server key exchange (12):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (IN), TLS handshake, Server finished (14):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (OUT), TLS change cipher, Client hello (1):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (OUT), TLS handshake, Finished (20):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (IN), TLS handshake, Server hello (2):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (IN), TLS handshake, Certificate (11):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (IN), TLS handshake, Server key exchange (12):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (IN), TLS handshake, Server finished (14):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (OUT), TLS change cipher, Client hello (1):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (OUT), TLS handshake, Finished (20):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (IN), TLS change cipher, Client hello (1):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: TLSv1.2 (IN), TLS handshake, Finished (20):
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: ALPN, server did not agree to a protocol
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: Server certificate:
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: subject: CN=sourcefinder.theskynet.org
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: start date: Aug 15 22:35:00 2017 GMT
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: expire date: Nov 13 22:35:00 2017 GMT
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: subjectAltName: sourcefinder.theskynet.org matched
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Info: SSL certificate verify ok.
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Sent header to server: GET /duchamp/download/32f/7_askap_cube_24_53_36.tar.gz HTTP/1.1
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Sent header to server: Host: sourcefinder.theskynet.org
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.6.33)
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Sent header to server: Accept: */*
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Sent header to server: Content-Type: application/x-www-form-urlencoded
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Sent header to server: Accept-Language: en_GB
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Sent header to server:
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (IN), TLS change cipher, Client hello (1):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (IN), TLS handshake, Finished (20):
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: ALPN, server did not agree to a protocol
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: Server certificate:
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: subject: CN=sourcefinder.theskynet.org
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: start date: Aug 15 22:35:00 2017 GMT
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: expire date: Nov 13 22:35:00 2017 GMT
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: subjectAltName: sourcefinder.theskynet.org matched
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: SSL certificate verify ok.
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Sent header to server: GET /duchamp/download/SofiaVM_1.3.vdi.gz HTTP/1.1
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Sent header to server: Host: sourcefinder.theskynet.org
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.6.33)
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Sent header to server: Accept: */*
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Sent header to server: Content-Type: application/x-www-form-urlencoded
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Sent header to server: Accept-Language: en_GB
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Sent header to server:
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Received header from server: HTTP/1.1 200 OK
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Received header from server: Date: Wed, 27 Sep 2017 13:05:34 GMT
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Received header from server: Server: Apache/2.2.32 (Amazon)
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Received header from server: Last-Modified: Wed, 27 Sep 2017 07:08:34 GMT
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Received header from server: ETag: "202305-eb6c27-55a2675c12993"
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Received header from server: Accept-Ranges: bytes
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Received header from server: Content-Length: 15428647
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Received header from server: Connection: close
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Received header from server: Content-Type: application/x-gzip
27/09/2017 14:05:33 | duchamp | [http] [ID#638] Received header from server:
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Received header from server: HTTP/1.1 403 Forbidden
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Received header from server: Date: Wed, 27 Sep 2017 13:05:34 GMT
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Received header from server: Server: Apache/2.2.32 (Amazon)
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Received header from server: Content-Length: 330
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Received header from server: Connection: close
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Received header from server: Content-Type: text/html; charset=iso-8859-1
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Received header from server:
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: Closing connection 1097
27/09/2017 14:05:33 | duchamp | [http] [ID#637] Info: TLSv1.2 (OUT), TLS alert, Client hello (1):
27/09/2017 14:05:34 | duchamp | Giving up on download of SofiaVM_1.3.vdi: permanent HTTP error
ID: 791 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Dr Who Fan
Avatar

Send message
Joined: 10 Feb 17
Posts: 3
Credit: 15,149
RAC: 42
Message 792 - Posted: 27 Sep 2017, 13:44:44 UTC
Last modified: 27 Sep 2017, 13:45:14 UTC

Also getting DOWNLOAD ERROR:
9/27/2017 08:36:05 | duchamp | Giving up on download of SofiaVM_1.3.vdi: permanent HTTP error


Stderr output
<core_client_version>7.8.2</core_client_version>
<![CDATA[
<message>
app_version download error: couldn't get input files:
<file_xfer_error>
<file_name>SofiaVM_1.3.vdi</file_name>
<error_code>-224 (permanent HTTP error)</error_code>
<error_message>permanent HTTP error</error_message>
</file_xfer_error>

</message>
]]>


ID: 792 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Yavanius
Avatar

Send message
Joined: 12 Feb 17
Posts: 53
Credit: 29,911
RAC: 51
Message 794 - Posted: 27 Sep 2017, 23:18:29 UTC
Last modified: 27 Sep 2017, 23:18:55 UTC

It's also leaving unworkable WUs because it cannot download the executable...
ID: 794 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mmonnin

Send message
Joined: 28 May 17
Posts: 5
Credit: 6,550
RAC: 13
Message 795 - Posted: 28 Sep 2017, 0:00:27 UTC - in response to Message 794.  

It's also leaving unworkable WUs because it cannot download the executable...


Shouldn't that be in the VM .vdi?

Stopped getting work due to the download errors.
ID: 795 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Yavanius
Avatar

Send message
Joined: 12 Feb 17
Posts: 53
Credit: 29,911
RAC: 51
Message 796 - Posted: 28 Sep 2017, 4:30:10 UTC - in response to Message 795.  
Last modified: 28 Sep 2017, 4:32:19 UTC

That's what I meant.

They clear out eventually. They clear out immediately if you do an Update. The server keeps sending work but since the VDI won't download...

Somebody get a hammer... American computers Russian computers... all made in same place...
ID: 796 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Dirk Broer

Send message
Joined: 22 Feb 17
Posts: 5
Credit: 18,260
RAC: 61
Message 799 - Posted: 28 Sep 2017, 22:32:06 UTC - in response to Message 790.  

You don't have permission to access /duchamp/download/SofiaVM_1.3.vdi on this server


so the file is present, but has the wrong attributes or owner



And this is still the case
ID: 799 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Yavanius
Avatar

Send message
Joined: 12 Feb 17
Posts: 53
Credit: 29,911
RAC: 51
Message 801 - Posted: 28 Sep 2017, 23:24:33 UTC - in response to Message 799.  

Damn Cardissan systems!
ID: 801 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 167
Credit: 7,636
RAC: 1
Message 808 - Posted: 3 Oct 2017, 23:36:04 UTC
Last modified: 3 Oct 2017, 23:45:53 UTC

Ah damn that's annoying..
Turns out, as people have identified, the VM image was stored in BOINC's download directory without read permissions, so the web server couldn't serve it.
The permissions should be fixed now, and the WU's should start flowing.

Last week's been a bit of a waste. Sorry about that, I should've checked the forums earlier. I've been rather busy.

Good news though, the results are now loading correctly in to the database!
ID: 808 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Yavanius
Avatar

Send message
Joined: 12 Feb 17
Posts: 53
Credit: 29,911
RAC: 51
Message 814 - Posted: 4 Oct 2017, 5:33:07 UTC - in response to Message 808.  
Last modified: 4 Oct 2017, 5:33:41 UTC

Ah damn that's annoying..
Turns out, as people have identified, the VM image was stored in BOINC's download directory without read permissions, so the web server couldn't serve it.
The permissions should be fixed now, and the WU's should start flowing.

Last week's been a bit of a waste. Sorry about that, I should've checked the forums earlier. I've been rather busy.

Good news though, the results are now loading correctly in to the database!


It's okay, I understand learning Cardissan is difficult and their computer systems are even worse...

But still...the spice...err work units, must flow....
ID: 814 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mmonnin

Send message
Joined: 28 May 17
Posts: 5
Credit: 6,550
RAC: 13
Message 817 - Posted: 4 Oct 2017, 22:50:31 UTC - in response to Message 814.  

I got some tasks last night after the .vdi permissions was updated. Most of the tasks are at 100% but have not completed with BM saying Waiting to run, Suspended: VM job unmanageable, restarting later. 7 are like that at 100%, 2 completed, neither are validated yet.
ID: 817 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Bill F

Send message
Joined: 9 Feb 17
Posts: 18
Credit: 47,612
RAC: 20
Message 818 - Posted: 6 Oct 2017, 17:38:22 UTC

You are moving froward we have running SoFIA work units. Small batch of 36 WU's 23 completed for Credit, 6 WU's marked Completed Invalid and 7 WU's failed with Errors.

All done on a Windows 7 AMD box using 7.8.2 and VBox 5.1.26

Bill
Dallas TX
ID: 818 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 167
Credit: 7,636
RAC: 1
Message 821 - Posted: 10 Oct 2017, 0:51:21 UTC - in response to Message 818.  

Good to hear. I'm glad things are running smoother again.
The fairly large number of errors and invalid work units is a bit troubling though...
ID: 821 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mmonnin

Send message
Joined: 28 May 17
Posts: 5
Credit: 6,550
RAC: 13
Message 823 - Posted: 11 Oct 2017, 11:41:44 UTC - in response to Message 821.  

Good to hear. I'm glad things are running smoother again.
The fairly large number of errors and invalid work units is a bit troubling though...


There would be a lot less if it were not a vbox app...
ID: 823 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Phil
Avatar

Send message
Joined: 9 Feb 17
Posts: 14
Credit: 74,948
RAC: 40
Message 824 - Posted: 11 Oct 2017, 14:42:05 UTC - in response to Message 823.  

Hmm, I'd be interested to know what percent of failed jobs can be attributed to vbox errors.
A problem of course is if a user hits Abort, BOINC doesnt record what was happening up to then.
ID: 824 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : News : SoFiA Beta Update 4


©2017 ICRAR