class Grit::Process
Override the Grit::Process class's popen4 and waitpid methods to work around various quirks in JRuby.
Grit::Process includes logic for executing child processes and reading/writing from their standard input, output, and error streams.
Create an run a process to completion:
>> process = Grit::Process.new(['git', '--help'])
Retrieve stdout or stderr output:
>> process.out => "usage: git [--version] [--exec-path[=GIT_EXEC_PATH]]\n ..." >> process.err => ""
Check process exit status information:
>> process.status => #<Process::Status: pid=80718,exited(0)>
Grit::Process is designed to take all input in a single string and provides all output as single strings. It is therefore not well suited to streaming large quantities of data in and out of commands.
Q: Why not use popen3 or hand-roll fork/exec code?
-
It's more efficient than popen3 and provides meaningful process hierarchies because it performs a single fork/exec. (popen3 double forks to avoid needing to collect the exit status and also calls Process::detach which creates a Ruby Thread!!!!).
-
It's more portable than hand rolled pipe, fork, exec code because fork(2) and exec(2) aren't available on all platforms. In those cases, Grit::Process falls back to using whatever janky substitutes the platform provides.
-
It handles all max pipe buffer hang cases, which is non trivial to implement correctly and must be accounted for with either popen3 or hand rolled fork/exec code.
Constants
- BUFSIZE
Maximum buffer size for reading
- FakeStatus
JRuby always raises ECHILD on pids returned from its IO.popen4 method for some reason. Return a fake Process::Status object.
Attributes
All data written to the child process's stderr stream as a String.
All data written to the child process's stdout stream as a String.
Total command execution time (wall-clock time)
A Process::Status object with information on how the child exited.
Public Class Methods
Create and execute a new process.
argv - Array of [command, arg1, …] strings to use as the new
process's argv. When argv is a String, the shell is used to interpret the command.
env - The new process's environment variables. This is merged with
the current environment as if by ENV.merge(env).
options - Additional options:
:input => str to write str to the process's stdin. :timeout => int number of seconds before we given up. :max => total number of output bytes A subset of Process:spawn options are also supported on all platforms: :chdir => str to start the process in different working dir.
Returns a new Process instance that has already executed to completion. The out, err, and status attributes are immediately available.
# File lib/grit/process.rb, line 58 def initialize(argv, env={}, options={}) @argv = argv @env = env @options = options.dup @input = @options.delete(:input) @timeout = @options.delete(:timeout) @max = @options.delete(:max) @options.delete(:chdir) if @options[:chdir].nil? exec! end
Public Instance Methods
Use JRuby's built in IO.popen4 but emulate the special spawn env and options arguments as best we can.
# File lib/grit/jruby.rb, line 9 def popen4(*argv) env = (argv.shift if argv[0].is_a?(Hash)) || {} opt = (argv.pop if argv[-1].is_a?(Hash)) || {} # emulate :chdir option if opt[:chdir] previous_dir = Dir.pwd Dir.chdir(opt[:chdir]) else previous_dir = nil end # emulate :env option if env.size > 0 previous_env = ENV ENV.merge!(env) else previous_env = nil end pid, stdin, stdout, stderr = IO.popen4(*argv) ensure ENV.replace(previous_env) if previous_env Dir.chdir(previous_dir) if previous_dir end
Determine if the process did exit with a zero exit status.
# File lib/grit/process.rb, line 84 def success? @status && @status.success? end
# File lib/grit/jruby.rb, line 38 def waitpid(pid) ::Process::waitpid(pid) $? rescue Errno::ECHILD FakeStatus.new(pid, 0, true, true) end
Private Instance Methods
Execute command, write input, and read output. This is called immediately when a new instance of this object is initialized.
# File lib/grit/process.rb, line 91 def exec! # when argv is a string, use /bin/sh to interpret command argv = @argv argv = ['/bin/sh', '-c', argv.to_str] if argv.respond_to?(:to_str) # spawn the process and hook up the pipes pid, stdin, stdout, stderr = popen4(@env, *(argv + [@options])) # async read from all streams into buffers @out, @err = read_and_write(@input, stdin, stdout, stderr, @timeout, @max) # grab exit status @status = waitpid(pid) rescue Object => boom [stdin, stdout, stderr].each { |fd| fd.close rescue nil } if @status.nil? ::Process.kill('TERM', pid) rescue nil @status = waitpid(pid) rescue nil end raise end
Start a select loop writing any input on the child's stdin and reading any output from the child's stdout or stderr.
input - String input to write on stdin. May be nil. stdin - The write side IO object for the child's stdin stream. stdout - The read side IO object for the child's stdout stream. stderr - The read side IO object for the child's stderr stream. timeout - An optional Numeric specifying the total number of seconds
the read/write operations should occur for.
Returns an [out, err] tuple where both elements are strings with all
data written to the stdout and stderr streams, respectively.
Raises TimeoutExceeded when all data has not been read / written within
the duration specified in the timeout argument.
Raises MaximumOutputExceeded when the total number of bytes output
exceeds the amount specified by the max argument.
# File lib/grit/process.rb, line 141 def read_and_write(input, stdin, stdout, stderr, timeout=nil, max=nil) input ||= '' max = nil if max && max <= 0 out, err = '', '' offset = 0 timeout = nil if timeout && timeout <= 0.0 @runtime = 0.0 start = Time.now writers = [stdin] readers = [stdout, stderr] t = timeout while readers.any? || writers.any? ready = IO.select(readers, writers, readers + writers, t) raise TimeoutExceeded if ready.nil? # write to stdin stream ready[1].each do |fd| begin boom = nil size = fd.write_nonblock(input) input = input[size, input.size] rescue Errno::EPIPE => boom rescue Errno::EAGAIN, Errno::EINTR end if boom || input.size == 0 stdin.close writers.delete(stdin) end end # read from stdout and stderr streams ready[0].each do |fd| buf = (fd == stdout) ? out : err begin buf << fd.readpartial(BUFSIZE) rescue Errno::EAGAIN, Errno::EINTR rescue EOFError readers.delete(fd) fd.close end end # keep tabs on the total amount of time we've spent here @runtime = Time.now - start if timeout t = timeout - @runtime raise TimeoutExceeded if t < 0.0 end # maybe we've hit our max output if max && ready[0].any? && (out.size + err.size) > max raise MaximumOutputExceeded end end [out, err] end
Spawn a child process, perform IO redirection and environment prep, and return the running process's pid.
This method implements a limited subset of Ruby 1.9's Process::spawn. The idea is that we can just use that when available, since most platforms will eventually build in special (and hopefully good) support for it.
env - Hash of { name => val } environment variables set in the child process. argv - New process's argv as an Array. When this value is a string, the command may be run through the system /bin/sh or options - Supports a subset of Process::spawn options, including: :chdir => str to change the directory to str in the child FD => :close to close a file descriptor in the child :in => FD to redirect child's stdin to FD :out => FD to redirect child's stdout to FD :err => FD to redirect child's stderr to FD
Returns the pid of the new process as an integer. The process exit status must be obtained using Process::waitpid.
# File lib/grit/process.rb, line 221 def spawn(env, *argv) options = (argv.pop if argv[-1].kind_of?(Hash)) || {} fork do # { fd => :close } in options means close that fd options.each { |k,v| k.close if v == :close && !k.closed? } # reopen stdin, stdout, and stderr on provided fds STDIN.reopen(options[:in]) STDOUT.reopen(options[:out]) STDERR.reopen(options[:err]) # setup child environment env.each { |k, v| ENV[k] = v } # { :chdir => '/' } in options means change into that dir ::Dir.chdir(options[:chdir]) if options[:chdir] # do the deed ::Kernel::exec(*argv) exit! 1 end end