|
|
|
|
|
|
| Author |
Message |
Chris F.A. Johnson
*nix forums Guru
Joined: 20 Feb 2005
Posts: 2268
|
 Posted: Thu Jul 20, 2006 7:57 pm Post subject:
Re: Multiple consumers scenario
|
|
|
On 2006-07-20, mrs wrote:
| Quote: | Sorry Chris - I thought that replying to your post was enough for you
to understand the context.
|
Please remember that you are posting to a Usenet newsgroup, not
Google Groups. There is no guarantee that the previous post is
still (or ever was) visible. If it was a reader may have read
hundreds of other posts since, and there is a very good chance
that the details will have been forgotten. Reading the previous
post[s] may or may not be a simple operation. It should be
unnecessary.
Also, don't top post. Put your reply after the appropriate text.
When posting from GG, please make sure your lines of code are
short enough that they don't wrap. That makes the code invalid. It
may be easy to fix (it is in this case), but often it isn't, and
most people will not take the time to decipher it,
| Quote: | on its first instance, the script is supposed to create a producer that
feeds a fifo. Then it recursively call itself to process one line at a
time until the list is finished.
|
What problems are you having?
What is the purpose of the script?
What is it supposed to do?
What input does it expect?
What output do you want?
| Quote: | -------------------------------------------------------------------------------------------------
#!/bin/bash
max_session=5
my_wd=$(pwd)
if [ -z $1 ] ; then
my_session=1
else
my_session=$(( $1 + 1 ))
fi
create_producer(){
if [ ! -p $my_wd/iofifo ] ; then
mkfifo $my_wd/iofifo
fi
while read line ; do echo $line > $my_wd/iofifo ; done < liste &
}
init_session(){
# session initialisation w/ the processing server
echo "$my_session : init_session"
}
process_line(){
# arg : $1 : line to process
echo "my session processing : $1"
}
close_session(){
echo "[7m$my_session : close_session[0m"
exit
}
fct_cleanup(){
if [ $my_session -ne 1 ]; then
kill -s SIGUSR1 $PPID
fi
close_session
}
trap "fct_cleanup" SIGHUP SIGINT SIGQUIT SIGILL SIGABRT SIGFPE SIGUSR1
SIGUSR2 SIGPIPE SIGTERM
read_iofifo(){
if [ -f $my_pwd/all_lines_processed ] ; then
kill -s SIGUSR1 $PPID
fi
read line < iofifo
if [ $line == 'Finish' ] ; then # Finish is the last file's line to
process
kill -s SIGUSR1 $$
else
process_line $line
fi
}
if [ $my_session == 1 ] ; then
create_producer
fi
echo "Hello - my pid is $$"
echo "my parent is $PPID"
if [[ $my_session -lt $max_session ]] ; then
$0 $my_session $my_duty_max $_duty_marker
fi
init_session
while read myline ; do
process_line $myline
done < <(read_iofifo)
-------------------------------------------------------------------------------------------------
Thanks for your help
Chris F.A. Johnson wrote:
On 2006-07-20, mrs wrote:
The very basic (and not functional) code would be this :
http://pastebin.ca/93246
Code to do what? Please include context in your follow-ups.
|
--
Chris F.A. Johnson, author <http://cfaj.freeshell.org>
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
===== My code in this post, if any, assumes the POSIX locale
===== and is released under the GNU General Public Licence |
|
| Back to top |
|
 |
mrs
*nix forums beginner
Joined: 20 Jul 2006
Posts: 3
|
| Posted: Thu Jul 20, 2006 9:57 am Post subject:
Re: Multiple consumers scenario
|
|
|
Sorry Chris - I thought that replying to your post was enough for you
to understand the context.
on its first instance, the script is supposed to create a producer that
feeds a fifo. Then it recursively call itself to process one line at a
time until the list is finished.
| Quote: | -------------------------------------------------------------------------------------------------
#!/bin/bash |
max_session=5
my_wd=$(pwd)
if [ -z $1 ] ; then
my_session=1
else
my_session=$(( $1 + 1 ))
fi
create_producer(){
if [ ! -p $my_wd/iofifo ] ; then
mkfifo $my_wd/iofifo
fi
while read line ; do echo $line > $my_wd/iofifo ; done < liste &
}
init_session(){
# session initialisation w/ the processing server
echo "$my_session : init_session"
}
process_line(){
# arg : $1 : line to process
echo "my session processing : $1"
}
close_session(){
echo "[7m$my_session : close_session[0m"
exit
}
fct_cleanup(){
if [ $my_session -ne 1 ]; then
kill -s SIGUSR1 $PPID
fi
close_session
}
trap "fct_cleanup" SIGHUP SIGINT SIGQUIT SIGILL SIGABRT SIGFPE SIGUSR1
SIGUSR2 SIGPIPE SIGTERM
read_iofifo(){
if [ -f $my_pwd/all_lines_processed ] ; then
kill -s SIGUSR1 $PPID
fi
read line < iofifo
if [ $line == 'Finish' ] ; then # Finish is the last file's line to
process
kill -s SIGUSR1 $$
else
process_line $line
fi
}
if [ $my_session == 1 ] ; then
create_producer
fi
echo "Hello - my pid is $$"
echo "my parent is $PPID"
if [[ $my_session -lt $max_session ]] ; then
$0 $my_session $my_duty_max $_duty_marker
fi
init_session
while read myline ; do
process_line $myline
done < <(read_iofifo)
| Quote: | -------------------------------------------------------------------------------------------------
|
Thanks for your help
Chris F.A. Johnson wrote:
| Quote: | On 2006-07-20, mrs wrote:
The very basic (and not functional) code would be this :
http://pastebin.ca/93246
Code to do what? Please include context in your follow-ups. |
|
|
| Back to top |
|
|
Chris F.A. Johnson
*nix forums Guru
Joined: 20 Feb 2005
Posts: 2268
|
| Posted: Thu Jul 20, 2006 9:36 am Post subject:
Re: Multiple consumers scenario
|
|
|
On 2006-07-20, mrs wrote:
Code to do what? Please include context in your follow-ups.
Post the code here unless it is extremely long.
| Quote: | I'll try to implement the lock/edit/unlock case which look interesting
(Thanks ANdrei)
|
--
Chris F.A. Johnson, author <http://cfaj.freeshell.org>
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
===== My code in this post, if any, assumes the POSIX locale
===== and is released under the GNU General Public Licence |
|
| Back to top |
|
|
mrs
*nix forums beginner
Joined: 20 Jul 2006
Posts: 3
|
| Posted: Thu Jul 20, 2006 9:30 am Post subject:
Re: Multiple consumers scenario
|
|
|
The very basic (and not functional) code would be this :
http://pastebin.ca/93246
I'll try to implement the lock/edit/unlock case which look interesting
(Thanks ANdrei)
Regards
mrs |
|
| Back to top |
|
|
aryzhov@spasu.net
*nix forums addict
Joined: 17 Feb 2005
Posts: 64
|
| Posted: Thu Jul 20, 2006 8:47 am Post subject:
Re: Multiple consumers scenario
|
|
|
I'd definitely prefere to implement this i C.
In shell,I see no option but using either a flag (lock) file,
or a master (frontend) process.
1. Each client checks the existance of lock file,
creates lock file,
takes a line
edits the file (removes a line), with "ed", for instance
removes lock
2. Each client creates 2 pipe files with unique names;
master process is aware of those names, for instance,
all pipe files may live in /tmp/pipe_in/$consumer-pid
and /tmp/pipe_out/$consumer-pid
As soon as consumer wants a line, he puts a unique
sequence number (call it "ticket") into his pipe-in
and goes on reading his pipe-out.
Master process reads a line, edits the file,
(or just reads the master pipe), and pools the pipe-ins
for "tickets". Feeds the line to the pipe_out of the clonsumer
whose
ticket has been just taken. Next line. Next consumer. Not ready?
Next.
Depending on how fast the consumers are, you may wish to awake the
master process with signals, rather than looping it in dumb pooling.
I doubt that master written in shell, can distinguish who has sent
the signal, otherwise picking the correct pipe to feed next line to,
would be much easier, and you wouldn't need the "ticket" pipes at all.
Regards,
Andrei |
|
| Back to top |
|
|
Krimgelas
*nix forums beginner
Joined: 22 Jun 2005
Posts: 35
|
| Posted: Thu Jul 20, 2006 8:46 am Post subject:
Re: Multiple consumers scenario
|
|
|
mrs wrote:
| Quote: | Hello
I'm currently encountering difficulties to write a mutliple consumers
bash snippet.
The problem is the following : I want multiple scripts that consumes a
file line by line . Each line of the file is consumed only once. Each
consumer is autonomous and has no idea of the other consumers'
existence. Each consumer consumes only one line at a time. The consumer
is persistent between two consumptions until there is no more line to
consume.
Does someone know how to achieve this ? I tried using fifo but I had
very poor results.
Thanks for your kind help
regard,
mrs
Sounds like a homework assignment...  |
|
|
| Back to top |
|
|
Chris F.A. Johnson
*nix forums Guru
Joined: 20 Feb 2005
Posts: 2268
|
| Posted: Thu Jul 20, 2006 7:59 am Post subject:
Re: Multiple consumers scenario
|
|
|
On 2006-07-20, mrs wrote:
| Quote: | Hello
I'm currently encountering difficulties to write a mutliple consumers
bash snippet.
The problem is the following : I want multiple scripts that consumes a
file line by line . Each line of the file is consumed only once. Each
consumer is autonomous and has no idea of the other consumers'
existence. Each consumer consumes only one line at a time. The consumer
is persistent between two consumptions until there is no more line to
consume.
Does someone know how to achieve this ? I tried using fifo but I had
very poor results.
|
Perhaps is you post what you tried, it will give us an idea of
what you are trying to do.
--
Chris F.A. Johnson, author <http://cfaj.freeshell.org>
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
===== My code in this post, if any, assumes the POSIX locale
===== and is released under the GNU General Public Licence |
|
| Back to top |
|
|
mrs
*nix forums beginner
Joined: 20 Jul 2006
Posts: 3
|
| Posted: Thu Jul 20, 2006 7:37 am Post subject:
Multiple consumers scenario
|
|
|
Hello
I'm currently encountering difficulties to write a mutliple consumers
bash snippet.
The problem is the following : I want multiple scripts that consumes a
file line by line . Each line of the file is consumed only once. Each
consumer is autonomous and has no idea of the other consumers'
existence. Each consumer consumes only one line at a time. The consumer
is persistent between two consumptions until there is no more line to
consume.
Does someone know how to achieve this ? I tried using fifo but I had
very poor results.
Thanks for your kind help
regard,
mrs |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Thu Dec 04, 2008 3:36 am | All times are GMT
|
|
Mortgages | Loan | Home Loan | Cheap mp3 players | Credit Card Consolidation
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
Programming
»
shell
