What is Sourcefinder?

Sourcefinder is a BOINC based distributed computing project that aims to provide a means to test the effectiveness of various astronomical sourcefinding applications. Sourcefinding applications are designed to process ‘cubes’ of data that contain radio sources and identify the location of the sources within the cube.
We’re currently testing the effectiveness of two different sourcefinding applications: Duchamp and SoFiA.

The effectiveness of a sourcefinding application can be determined by comparing the number of correct sources found within a cube to the number of incorrect sources found.
In order to provide a fair comparison between sourcefinding applications, we’re currently getting each application to process the same sections of a large cube known as the DINGO cube. DINGO is a simualated cube of data that contains a known catalogue of valid sources, with the addition of various levels of simulated noise. The total size of the DINGO cube is around 700GB.

This is where BOINC comes in. The huge cube is subdivded in to much smaller cubelets, around 10MB to 100MB each, that are processed by the computers of BOINC users. Each user runs one of the sourcefinding applications on their local machine, processes the small cubelet, then reports any sources found within the cubelet back to the project.
Once we’ve processed the full cube through the Duchamp and SoFiA sourcefinders, the results will be compared by a researcher, and a paper will be written on the comparitive performance of these sourcefinders.
We also have the prospect of processing cube data generated by ASKAP in the near future, as ASKAP has almost completed processing a large cube of sources.

The Sourcefinder project is run as part of the wider theSkyNet project.

We welcome any sort of feedback, advice, or bug reports. You can either make a post on the forums or send us an email at icrar.tsn.website@gmail.com . We're happy to hear from you.

Join Sourcefinder

User of the Day

User profile Profile WhiteWulfe
Hi, I'm WhiteWulfe, a member of Overclock.net's BOINC team that hails from Northern Alberta, Canada who also happens to be in his mid-thirties. I enjoy...

News

SoFiA 100MB Progress Update
Hello Everyone,

Just a quick progress update on the processing of our 100MB cubelets.
So far we've successfully processed 5 / 16 cubelet sets, and we're processing about 1 set per week.
The results database is looking pretty good too. I'm not seeing too many validation errors (about one every 30 or so results), so that's certainly promising.

Just a note that you'll only receive 100MB cubelets if you opt in to receiving them via checking the "Run test applications?" checkbox on the Sourcefinder preferences page. This is because the 100MB cubelets are significantly more CPU, memory, and disk intensive than the previous cubelets.

Thanks,
Sam
21 Mar 2018, 1:13:12 UTC · Discuss


theSkyNet.org back up
Hi Everyone,

Just a quick note to say I've revived theSkyNet.org web servers again.

Regards,
Sam
13 Mar 2018, 23:51:55 UTC · Discuss


SoFiA Update 8 and some changes at ICRAR
Hi everyone,

There's been a lot going on over the last week at ICRAR, and I've got a lot to update you on.

I was originally contracted by Outreach to work more or less exclusively on theSkyNet. As of March this year, my contract is being moved over to DIA (Data intensive astronomy), who want to move me to work on other projects.

I met with the head of DIA last week and it's been decided that after the 100MB workunits are all done done, I'll need to wrap up Sourcefinder and organise all of the data we've crunched so far in preparation for analysis.

So bearing this in mind, my new plan is to continue running 100MB workunits through the sofiabeta app, as everything is more less working there as it is. sofiabeta will remain a beta app so people can opt in / opt out as they please, as the 100MB workunits are significantly more intensive than the 10MB workunits,

Then, once we've completed the 100MB workunits, I'll spend some time working on a public contributers page that'll list everyone who has contributed to Sourcefinder. I'll also try and ensure that a link to this page is present in the final paper analysing the results. Last I checked they were generally against the idea of mentioning a list of contributers in the paper, but they thought that providing a link to a web page that listed the contributers would be fine.

As for the ASKAP data that was promised to me months ago, I'm still not sure of its current status. Once the 100MB workunits are done, I'll see if I can chase it down and get an update as to whether they want us to process it or not.

I also wanted to write a results viewer that would allow users to search for sources they've found and view them, but it doesn't look like I'll be able to do that anymore if I'm being moved to other projects. Sorry about that.

As for SoFiA, I'm looking at increasing the rsc_memory_bound for the 100MB workunits to be 4GB, meaning computers with less than 4GB of memory wont be given these workunits. Thanks to LumenDan for bringing this up.
I'll keep pushing out at least one set of workunits per week, but depending on the disk space of the server, I might be able to push out two. We'll see.

If there are any clarifications anyone needs, feel free to ask. I'll try to answer anything.

Regards,
Sam
28 Feb 2018, 2:33:41 UTC · Discuss


New Sofia Beta App (Opt-In)
Hello everyone,

After the disaster that was last week, I took the advice of LumenDan and I've created a new app, sofiabeta. I'll be using this app in the future to trial new client versions and changes before using the main sofia app.

The sofiabeta app is opt in, so if you don't want to run it, you don't have to. You can opt into / out of running test applications here.

There's currently a set of test workunits waiting, which I've split into 6 runs to ease the ridiculous computation times.

I hope this goes smoothly.

Sam
31 Jan 2018, 4:58:04 UTC · Discuss


SoFiA Production Update 7 { Cancelled - See Forum for Details }
So the SoFiA devs managed to work out a fix to those extremely large memory requirements I mentioned last week, and I've been able to drop the VM memory requirements down to 2GB per VM. I know that's still a lot more than they were before, but it doesn't look like I'll be able to drop them any less without hitting some huge performance decreases (due to disk swapping).

The VM has been updated with this new version of SoFiA, and I've spend some extra time compacting it as far as possible. I managed to get the VM size below 500MB compressed!

The workunits from last week didn't release properly due to an issue with the workunit generator that I couldn't get around to fixing until now, but the test WUs should be flowing as I write. I'm interested to see how the system handles them.
24 Jan 2018, 5:09:09 UTC · Discuss


... more

News is available as an RSS feed   RSS


©2018 ICRAR