WATCHDOG

This is my Watchdog script to regulate the crawling scripts …..

Line 5-6 is to reset the scripts

Line 8 is to start the selenium server

Line 11 is my proxy port pointer … i user several ports ant to set the port I let the pointer iterate ….

Line 13-12 i count the phantomjs and main.pl

Line 15-16 is to remove the line break ….

Line 18-21 check the maximum values the start the script and let the pointer iterate ….

then sleep and redo the work ….

#!/usr/bin/perl
use strict;
use warnings;

`killall phantomjs`;
`killall main.pl`;

system('java -jar ./selenium-server-standalone-2.40.0.jar >> /dev/null 2>&1 &');
#`killall -s 9 phantomjs && sleep 1`;
#system('./phantomjs-1.9.7-linux-x86_64/bin/phantomjs --webdriver=8888 >> /dev/$
my $pp=10;
do{
my $count = `ps aux | grep phantomjs | grep -v grep | wc -l`//0;
my $skript = `ps aux | grep main.pl | grep -v grep | wc -l`//0;
$count  =~ s/n//og;
$skript =~ s/n//og;
print "Main : $count - Gesamt :$skript".$/;
if($count < 4 && $skript < 4){
system("./main.pl '88$pp' > /dev/null 2>&1 &");
++$pp;$pp =10 if($pp>=20);
}
sleep(20);
}while(1);

 

One thought on “WATCHDOG”

Leave a Reply

Your email address will not be published.

ten − five =