Real data coming soon!

Message boards : News : Real data coming soon!

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 138
Credit: 7,576
RAC: 0
Message 635 - Posted: 21 Jun 2017, 4:52:54 UTC
Last modified: 2 Aug 2017, 5:42:59 UTC

I've just arrived back from a meeting with Kirsten, and I have some extremely good news to share with everyone!

ASKAP has almost completed observations of its first ~500GB cube, and we're going to be processing it when it's done!
Running a sourcefinding application on ASKAP data hasn't been done yet, so this is going to be a world first.

While we're waiting for ASKAP to be done, we're going to be running the simulated super cube again, this time using a new sourcefinding application called SoFiA.
We're going to be looking at whether SoFiA is any better at finding sources than Duchamp is.

Once we have data from both Duchamp and SoFiA, I'll be working on cross matching the sources they found with the original sources catalogue, then we'll have a clear picture of which is the better sourcefinder.

I'm also told that there's a very high chance of a paper being written on the comparison between Duchamp and SoFiA as sourcefinders, so I'll be sure to keep you all up to date if I hear any more about it.

Finally, there's been a bit of a back end storage issue with some of the data we've processed, and I'll need to re-run a few sets of work units, so expect some more work to be coming fairly soon.

Edit: Please see my other post that expands on some of the things I've discussed in this one.
ID: 635 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
LumenDan

Send message
Joined: 9 Feb 17
Posts: 59
Credit: 118,297
RAC: 19
Message 636 - Posted: 21 Jun 2017, 8:37:55 UTC - in response to Message 635.  

That's great news.

Will SoFIA be a new project or will it run as a separation application on the existing SourceFinder project url?
ID: 636 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
adrianxw

Send message
Joined: 4 May 17
Posts: 32
Credit: 30,566
RAC: 1
Message 637 - Posted: 21 Jun 2017, 10:51:58 UTC

... my question also.
ID: 637 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
boinc-seti

Send message
Joined: 5 May 17
Posts: 2
Credit: 5,402
RAC: 2
Message 638 - Posted: 21 Jun 2017, 16:08:12 UTC

Mine as well.
ID: 638 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Euphoriabuzz

Send message
Joined: 4 Mar 17
Posts: 2
Credit: 24,012
RAC: 1
Message 639 - Posted: 21 Jun 2017, 17:32:41 UTC

Fab news. We wait in anticipation.
ID: 639 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 138
Credit: 7,576
RAC: 0
Message 640 - Posted: 21 Jun 2017, 23:56:24 UTC - in response to Message 636.  

I'm currently thinking of distributing it via this project, but under a different appid. I believe that'd be more convenient for everyone, because you wont all have to attach to a new project.
ID: 640 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
adrianxw

Send message
Joined: 4 May 17
Posts: 32
Credit: 30,566
RAC: 1
Message 641 - Posted: 22 Jun 2017, 6:13:07 UTC
Last modified: 22 Jun 2017, 6:59:43 UTC

What you should have is a single Skynet project and run duchamp, SoFiA and pogs under that banner.
ID: 641 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
boinc-seti

Send message
Joined: 5 May 17
Posts: 2
Credit: 5,402
RAC: 2
Message 643 - Posted: 22 Jun 2017, 15:32:29 UTC - in response to Message 641.  

Touché
ID: 643 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Yavanius
Avatar

Send message
Joined: 12 Feb 17
Posts: 39
Credit: 27,920
RAC: 5
Message 650 - Posted: 24 Jun 2017, 17:06:45 UTC - in response to Message 635.  
Last modified: 24 Jun 2017, 17:07:52 UTC

ASKAP has almost completed observations of its first ~500GB cube, and we're going to be processing it when it's done!
Running a sourcefinding application on ASKAP data hasn't been done yet, so this is going to be a world first.

While we're waiting for ASKAP to be done, we're going to be running the simulated super cube again, this time using a new sourcefinding application called SoFiA.
We're going to be looking at whether SoFiA is any better at finding sources than Duchamp is.



When you have a chance Sam, can you explain the overall mission of Skynet, how POGS and Duchamp fit in there, and clarify how they differ from each other (what is the intended focus of each)?
ID: 650 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey

Send message
Joined: 1 Apr 17
Posts: 1
Credit: 0
RAC: 0
Message 655 - Posted: 27 Jun 2017, 18:23:30 UTC - in response to Message 640.  

I'm currently thinking of distributing it via this project, but under a different appid. I believe that'd be more convenient for everyone, because you wont all have to attach to a new project.


+1000
ID: 655 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 138
Credit: 7,576
RAC: 0
Message 656 - Posted: 27 Jun 2017, 23:48:27 UTC - in response to Message 641.  

My main worry about trying to get POGS and Duchamp to run under the same project is that Sourcefinder requires virtualbox and POGS doesn't. There could be issues if I tried to merge them, as not everyone is able to run virtualbox on their machines.
ID: 656 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
adrianxw

Send message
Joined: 4 May 17
Posts: 32
Credit: 30,566
RAC: 1
Message 657 - Posted: 28 Jun 2017, 15:35:06 UTC - in response to Message 656.  
Last modified: 28 Jun 2017, 15:39:01 UTC

Fair point. I don't know if that creates a problem or not, I can see how it might, but then again, I can not say without further investigation. Whatever, do what you think is best. An obvious way forward is to make it an "opt in" subproject, then people who can, will, those who can't can ignore it.

There will be a few of us around to crunch whichever way. It is just the rapidly expanding number of seperate projects around makes for awkward running management.
ID: 657 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Tom_unoduetre

Send message
Joined: 9 Feb 17
Posts: 4
Credit: 9,291
RAC: 2
Message 658 - Posted: 29 Jun 2017, 7:40:45 UTC - in response to Message 657.  

You could check with the guys from LHC@home, they run different apps under the same project, most of them VB apps but also one (or two?) non virtualbox apps and it´s running fine. You can just select in the preferences which app you want to run.
ID: 658 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Dave Peachey

Send message
Joined: 9 Feb 17
Posts: 41
Credit: 765,771
RAC: 141
Message 659 - Posted: 29 Jun 2017, 8:04:30 UTC - in response to Message 658.  

You could check with the guys from LHC@home, they run different apps under the same project, most of them VB apps but also one (or two?) non virtualbox apps and it´s running fine. You can just select in the preferences which app you want to run.
The only problem I can forsee with this approach (as I find with LHC@home) is that, if you want to run two or more apps from one project in different proporitions/priorities on one computer, the preferences would be tricky (at best) to manage.

You might be able to do that, locally, via an <app_config> setup on the machine but not everyone wants to get "under the hood" to that extent and (as with LHC@home) that also generates lots of queries from the general user populace.

It's less of an issue running different apps with different priorities across several machines as you can use the different project profiles (home, school, work) to organise each machine's app priorities ... except that wouldn't work for people with four or more machines who might want a different balance of app priorities on every machine.

Personally, I think it's better to keep the POGS and Sourcefinder projects completely separate (notwithstanding having a consolidated "score" across both projects might be a nice thng to do) .. but that's just my opinion.

Dave
ID: 659 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 138
Credit: 7,576
RAC: 0
Message 660 - Posted: 29 Jun 2017, 23:58:17 UTC - in response to Message 659.  

I do want to eventually get around to consolidating scores from both POGS and Sourcefinder on theskynet.org, probably after I get the result visualization for Sourcefinder working.
ID: 660 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tito

Send message
Joined: 23 Jun 17
Posts: 1
Credit: 6,086
RAC: 4
Message 661 - Posted: 30 Jun 2017, 7:20:27 UTC

Consolidate scores? Same as at LHC projects?
ID: 661 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
LumenDan

Send message
Joined: 9 Feb 17
Posts: 59
Credit: 118,297
RAC: 19
Message 662 - Posted: 30 Jun 2017, 9:22:56 UTC - in response to Message 661.  

I think it is a good idea to keep as many applications as possible under one banner but I can see the benefit in keeping POGS and Source Finder separate to simplify the administration of Source Finder and keep the scope of your current project development under control.

If SoFiA proves to be a better Source Finder than Duchamp will it still be appropriate to use https://sourcefinder.theskynet.org/duchamp as the project URL?
If a more general URL is chosen it would avoid confusion when changing algorithms but changing the project URL is a big deal so if it is a consideration I think it should be done soon while the number of contributors is still quite low.

Is it possible to treat computers with VirtualBox as a Plan Class when managing a mix of applications? If so I wouldn't see it as a problem to mix native and virtual applications.
I would like to see some native compiled applications become available as the project matures, removing the restrictions of VirtualBox and allowing for architecture optimisations would improve the efficiency of calculation and further improve the appeal of the project. VirtualBox is great for testing new applications but I would hope that a proven algorithm could be compiled for all of the major architectures/operating systems over time.
ID: 662 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
MechanoRealist

Send message
Joined: 9 Feb 17
Posts: 2
Credit: 184,027
RAC: 42
Message 663 - Posted: 2 Jul 2017, 10:05:49 UTC - in response to Message 662.  

^ Seconding what LumenDan said.
ID: 663 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Sam
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Help desk expert

Send message
Joined: 9 Feb 17
Posts: 138
Credit: 7,576
RAC: 0
Message 670 - Posted: 4 Jul 2017, 23:55:02 UTC - in response to Message 662.  

When I was originally setting up this project, I thought the only sourcefinding application we'd be using would be Duchamp. It was only recently that SoFiA was even mentioned to me. I do agree though, a project url such as https://sourcefinder.theskynet.org/sourcefinder (similar to https://pogs.theskynet.org/pogs) would be desirable, as it removes any confusion about the duchamp part of the URL.

I also would like to be able to run Duchamp and SoFiA as standalone applications, but they both have a reliance on additional astronomy libraries. Now I could possibly try and get the source code of all of those libraries and just compile them together in to one executable, but I'm not sure how easy it'd be to get everything to play nicely. I had a brief look at doing something like this a few months ago, but decided it would probably take too much of my time (only being able to work 1 day a week on Sourcefinder is annoying at times).

For the future, though, I would like to remove the VirtualBox restriction.
ID: 670 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mmonnin

Send message
Joined: 28 May 17
Posts: 2
Credit: 6,013
RAC: 3
Message 673 - Posted: 6 Jul 2017, 1:19:18 UTC - in response to Message 670.  
Last modified: 6 Jul 2017, 1:20:51 UTC

For the future, though, I would like to remove the VirtualBox restriction.



YES!! I can't agree with this more, nor make that much bigger!! Compared to pogs this project is SO much smaller. VBox projects typically are as VBox is picky, take up a lot of space, use a lot of memory, and can be a PIA to get working right. Duchamp is newer, sure..but VBox plays a role in the smaller user base.
ID: 673 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · Next

Message boards : News : Real data coming soon!


©2017 ICRAR