I have a few going also but only 2 of my machine have some. It should pick up soom.

Ok, thanks. Still nothing coming through. Will just have to wait.

Anybody experiencing slow down whit this project?
One off my pc can't even get on the internet anymore.

If they fix there apps it would be a good idea. For now, It's not stable.The first 50% is ok but the last part is slowing down my system. Dont know if it's only me or if it does that with others.I'll see after checking it out. Is it running ok on your side?

I've never had a problem - how many simultaneous spiders are you running from within your network? If you run too many you may use up all of your bandwidth, especially if you're running any P2P programs at the same time...

20 at the moment. Is there a way to limit them? Thats on 1 machine. I have ultra high speed. It should not afect the internet.

Your Account-->DepSpid Preferences-->Edit Preferences

My account was defaulted to 2, I changed it to 10 and was getting the slow down you described. I now have it set to 4 and it seems ok.

Hope that helps!

Thanks will check it out now.

I've always had mine defaulted to 1 - but then again I've got lots of clients on a single network!

Just think of the internet and trying to open 2 or 3 webpages at once and you'll get a bit of a slowdown. With DepSpid leeching 20 at a time you're NIC/router/whatever is likely having a fit even if your available bandwidth is still OK. I know whenever I'm doing some hardcore leeching on one of my home systems it always locks up the network card until I stop whatever I'm doing (and I've even got a gigabit NIC).

Theres definitively something about those last few WU.
One, they take forever to run compared to the bunch before the maintenance.
Two, i dont have the best connection out there, but with these it fells like 56k all over again (Urgh!)

And thats with only 1 running, cant imagine you folks with 20

Here is a new one, I have one computer running 2 DepSpid WUs and it's regular WU. I'll be getting bonus points now.

Same here
"05/03/2008 1:48:05 AM|DepSpid|Scheduler request succeeded: got 2 new tasks"

DepSpid is back on-line with new wus!

Drat, thought i could catch up to some of you guys abit before the news came out

The only thing is that it's not wide open yet. Only received 1 wu and there out off work allready.
Maybe is going to be beter today.

The workunits have run out guys, after the feast comes the famine

And me down to my last four wus...it was nice while it lasted...

I miss those spiders

Update from the site forum.

Posted: Sun May 25, 2008 11:14 am Post subject: Availability of tasks

--------------------------------------------------------------------------------

As you may already have noticed, there either wasn't much work available during the last few weeks or the available tasks completed very fast. A few different things came together and finally resulted in this situation...

In mid 2007, I made the decision to split the spider application into two phases: the networking phase (phase-1) and a computational phase (phase-2). You can read more about that here: http://www.bjoernhenke.de/forum/viewtopic.php?t=547.

At the end of 2007, DepSpid started using a completely rewritten version of the spider application which uses in-memory databases instead of an sqlite table on the harddisk to dramatically lower harddisk activities and reduce cpu usage. This new version was a phase-1-only application without the computational part. But without the computational part, no new domains were being added to the backend database because this process relies on the phase-2 part. At the beginning, this didn't seemed to be a big problem because of a few million unprocessed domains in the database. But in April 2008, the remaining number of unprocessed domains reached almost zero.

At the current stage of the project, already successfully processed domains are not going to be reprocessed. But there is already a quite large amount of domains that failed being processed for the first time. The main reasons for this are errors like not being able to look-up a DNS name, difficulties to reach the domain for whatever reason or simply that the result of the task was never reported back to the project server. Two weeks ago, DepSpid started sending out tasks with domains that failed during DNS look-up. Because many of these domains truly doesn't exist, most of these tasks completed very fast with almost no network traffic being done and therefore not much credit being granted. Current tasks are now a mixture of domains failing before for different reasons instead of doing one reason after another. That's why most current tasks are behaving normally again.

But reprocessing failed domains won't give new tasks forever and preparing these tasks takes much more time than for normal tasks. In a few weeks, the project is going to be completely out of tasks until phase-2 processing can be re-established.

The first tests for a phase-2 application have already taken place but they have shown surprising results. It's currently unsure what these results will mean for the future steps of the project. I hope to be able to give more details about that within the next two weeks.

Yippee I got 5 spiders in one go instead of the usual one, hopefully there is a load of workunits coming our way

I got a few also on 3 boxes. Should bring my score ut a bit.

I just spotted this on the news feed from Depspid

Bjoern Henke
Site Admin


Age: 30
Joined: 09 Sep 2003
Posts: 499
Location: Hemmingen (Germany)
Posted: Sun Jul 27, 2008 12:40 pm Post subject: Phase-2 status

--------------------------------------------------------------------------------

As you most likely already have noticed, there is still no work available for DepSpid.
The reason is the phase-2 application development which isn't progressing as expected.

Before separating the two phases, the spider application took around 15-60 minutes of cpu time for phase-2 result calculation. This part was planed to be moved into a separate application. The original spider application made heavy usage of an sql database and spent much time with disk access. With the separation of the two phases, the sql backend was completely removed and superseded by in-memory operations. This change was expected to dramatically reduce the time needed to process a task. The original idea to accumulate for this was to bundle multiple phase-1 results together into a single phase-2 task. While testing the first versions of a phase-2 application, it became obvious, that even this way a task won't take more than a few minutes of cpu time. But the time for the server backend to reintegrate phase-2 results into phase-1 tasks was expected to be much longer than that. Therefore, it would have been much better for the server to do the calculations on its own and directly handle phase-1 tasks. That's why the original concept (and the already developed phase-2 application) was thrown away and a new concept had to be created.

The new concept is to directly use phase-1 results as tasks for the phase-2 application and move as much as possible from backend logics into the phase-2 application. This way, processing phase-1 tasks will take almost no server time and processing phase-2 results will reduce server usage to a minimum (result validation and database insertions only).

The newly designed phase-2 application is currently being developed and tested and will be ready for the first tests within August 2008. If you want to help testing this application, please PM me.

The new time plan is as follows:

From Now until 05th September: Further development and testing of phase-2 application
06th September - 14th September: Planned outage (~1 week)
15th September - 30th September: Phase-2 processing for all already completed phase-1 tasks
1st October - 12th October: Planned outage (~2 weeks)
Starting at 13th October: New phase-1 (and phase-2) tasks should be available to process
Beyond: Phase-3 development and testing will