socketpair-based popen experiment

Craig A. Berry · *nix forums Guru Wannabe Joined: 27 May 2005 Posts: 143

The most common cause of test failures on VMS is the insertion of extra
newlines in pipes. For more details, see the archives, but the gist of
it is that you never know whether

print 'a', 'b', 'c', "\n";

will give you the same thing as

print "abc\n";

when the output device is a pipe and autoflushing is enabled (which it
typically needs to be for inter-process communication). The first
example might well give you an extra newline somewhere in the middle.

So I took a swing at replacing our mailbox-based pipe implementation
with a socketpair-based implementation. Since sockets are
stream-oriented devices (unlike mailboxes, which are record-oriented),
it seemed like this at least had the potential to solve our problems.
There is a diff attached to this message that shows what I did. The
experiment failed, for reasons that I'll attempt to summarize. I used
the socketpair that came with the CRTL in v8.2, but if other obstacles
can be overcome, a homegrown socketpair might well work on older
versions of VMS.

I borrowed some infrastructure from the existing pipe implementation but
left it intact, protected by #ifdefs. I did get communication happening
through the socket-based pipes, but all carriage control was stripped.
Instead of getting the occasional spurious newline, I got none at all.
Somehow the sockets need to be convinced to leave the carriage control
information alone, but I was not able to come up with a way to do that.

Aside from the obvious fact that I didn't get it working, the main
drawbacks to using sockets come down to:

-- The TCP/IP Services documentation says that standard I/O is not
supported on sockets, and specifically fdopen() is not supported. Since
popen() returns a FILE pointer, you absolutely have to have that. The
calls to fdopen() succeed, but if the other stdio functions don't
properly handle the resulting FILE pointers, that may explain the loss
of carriage control.

-- Modifying the buffer size on a socket cannot be done without
privileges; you're stuck with 256 bytes, which is likely to be a serious
performance drag.

I'm not sure I've gained anything except a better understanding of the
challenge. I think the next thing to try is turning off all the
home-grown piping code and seeing if the new DECC$STREAM_PIPE feature
does us any good, though that unfortunately requires 8.2 and later.

BTW, as I understand it, most modern unices use shared memory for pipes.
Is there any particular reason we couldn't use global sections on VMS?

Google