runners

class invoke.runners.Runner(context: Context)

Partially-abstract core command-running API.

This class is not usable by itself and must be subclassed, implementing a number of methods such as start, wait and returncode. For a subclass implementation example, see the source code for Local.

在 1.0 版本加入.

__init__(context: Context) → None

Create a new runner with a handle on some Context.

参数:

context –

a Context instance, used to transmit default options and provide access to other contextualized information (e.g. a remote-oriented Runner might want a Context subclass holding info about hostnames and ports.)

备注

The Context given to Runner instances must contain default config values for the Runner class in question. At a minimum, this means values for each of the default Runner.run keyword arguments such as echo and warn.

抛出:

exceptions.ValueError – if not all expected default values are found in context.

context

The Context given to the same-named argument of __init__.

program_finished

A threading.Event signaling program completion.

Typically set after wait returns. Some IO mechanisms rely on this to know when to exit an infinite read loop.

read_chunk_size = 1000

How many bytes (at maximum) to read per iteration of stream reads.

input_sleep = 0.01

How many seconds to sleep on each iteration of the stdin read loop and other otherwise-fast loops.

warned_about_pty_fallback

Whether pty fallback warning has been emitted.

watchers: List['StreamWatcher']

A list of StreamWatcher instances for use by respond. Is filled in at runtime by run.

run(command: str, **kwargs: Any) → Result | None

Execute command, returning an instance of Result once complete.

By default, this method is synchronous (it only returns once the subprocess has completed), and allows interactive keyboard communication with the subprocess.

It can instead behave asynchronously (returning early & requiring interaction with the resulting object to manage subprocess lifecycle) if you specify asynchronous=True. Furthermore, you can completely disassociate the subprocess from Invoke’s control (allowing it to persist on its own after Python exits) by saying disown=True. See the per-kwarg docs below for details on both of these.

备注

All kwargs will default to the values found in this instance’s context attribute, specifically in its configuration’s run subtree (e.g. run.echo provides the default value for the echo keyword, etc). The base default values are described in the parameter list below.

参数:

• command (str) – The shell command to execute.

• asynchronous (bool) –

  When set to True (default False), enables asynchronous behavior, as follows:

  • Connections to the controlling terminal are disabled, meaning you will not see the subprocess output and it will not respond to your keyboard input - similar to hide=True and in_stream=False (though explicitly given (out|err|in)_stream file-like objects will still be honored as normal).

  • run returns immediately after starting the subprocess, and its return value becomes an instance of Promise instead of Result.

  • Promise objects are primarily useful for their join method, which blocks until the subprocess exits (similar to threading APIs) and either returns a final Result or raises an exception, just as a synchronous run would.

    • As with threading and similar APIs, users of asynchronous=True should make sure to join their Promise objects to prevent issues with interpreter shutdown.

    • One easy way to handle such cleanup is to use the Promise as a context manager - it will automatically join at the exit of the context block.

  在 1.4 版本加入.

• disown (bool) –

  When set to True (default False), returns immediately like asynchronous=True, but does not perform any background work related to that subprocess (it is completely ignored). This allows subprocesses using shell backgrounding or similar techniques (e.g. trailing &, nohup) to persist beyond the lifetime of the Python process running Invoke.

  备注

  If you’re unsure whether you want this or asynchronous, you probably want asynchronous!

  Specifically, disown=True has the following behaviors:

  • The return value is None instead of a Result or subclass.

  • No I/O worker threads are spun up, so you will have no access to the subprocess’ stdout/stderr, your stdin will not be forwarded, (out|err|in)_stream will be ignored, and features like watchers will not function.

  • No exit code is checked for, so you will not receive any errors if the subprocess fails to exit cleanly.

  • pty=True may not function correctly (subprocesses may not run at all; this seems to be a potential bug in Python’s pty.fork) unless your command line includes tools such as nohup or (the shell builtin) disown.

  在 1.4 版本加入.

• dry (bool) –

  Whether to dry-run instead of truly invoking the given command. See --dry (which flips this on globally) for details on this behavior.

  在 1.3 版本加入.

• echo (bool) –

  Controls whether run prints the command string to local stdout prior to executing it. Default: False.

  备注

  hide=True will override echo=True if both are given.

• echo_format –

  A string, which when passed to Python’s inbuilt .format method, will change the format of the output when run.echo is set to true.

  Currently, only {command} is supported as a parameter.

  Defaults to printing the full command string in ANSI-escaped bold.

• echo_stdin (bool) –

  Whether to write data from in_stream back to out_stream.

  In other words, in normal interactive usage, this parameter controls whether Invoke mirrors what you type back to your terminal.

  By default (when None), this behavior is triggered by the following:

  • Not using a pty to run the subcommand (i.e. pty=False), as ptys natively echo stdin to stdout on their own;

  • And when the controlling terminal of Invoke itself (as per in_stream) appears to be a valid terminal device or TTY. (Specifically, when isatty yields a True result when given in_stream.)

    备注

    This property tends to be False when piping another program’s output into an Invoke session, or when running Invoke within another program (e.g. running Invoke from itself).

  If both of those properties are true, echoing will occur; if either is false, no echoing will be performed.

  When not None, this parameter will override that auto-detection and force, or disable, echoing.

• encoding (str) – Override auto-detection of which encoding the subprocess is using for its stdout/stderr streams (which defaults to the return value of default_encoding).

• err_stream – Same as out_stream, except for standard error, and defaulting to sys.stderr.

• env (dict) –

  By default, subprocesses receive a copy of Invoke’s own environment (i.e. os.environ). Supply a dict here to update that child environment.

  For example, run('command', env={'PYTHONPATH': '/some/virtual/env/maybe'}) would modify the PYTHONPATH env var, with the rest of the child’s env looking identical to the parent.

  参见

  replace_env for changing ‘update’ to ‘replace’.

• fallback (bool) – Controls auto-fallback behavior re: problems offering a pty when pty=True. Whether this has any effect depends on the specific Runner subclass being invoked. Default: True.

• hide –

  Allows the caller to disable run’s default behavior of copying the subprocess’ stdout and stderr to the controlling terminal. Specify hide='out' (or 'stdout') to hide only the stdout stream, hide='err' (or 'stderr') to hide only stderr, or hide='both' (or True) to hide both streams.

  The default value is None, meaning to print everything; False will also disable hiding.

  备注

  Stdout and stderr are always captured and stored in the Result object, regardless of hide’s value.

  备注

  hide=True will also override echo=True if both are given (either as kwargs or via config/CLI).

• in_stream –

  A file-like stream object to used as the subprocess’ standard input. If None (the default), sys.stdin will be used.

  If False, will disable stdin mirroring entirely (though other functionality which writes to the subprocess’ stdin, such as autoresponding, will still function.) Disabling stdin mirroring can help when sys.stdin is a misbehaving non-stream object, such as under test harnesses or headless command runners.

• out_stream – A file-like stream object to which the subprocess’ standard output should be written. If None (the default), sys.stdout will be used.

• pty (bool) –

  By default, run connects directly to the invoked process and reads its stdout/stderr streams. Some programs will buffer (or even behave) differently in this situation compared to using an actual terminal or pseudoterminal (pty). To use a pty instead of the default behavior, specify pty=True.

  警告

  Due to their nature, ptys have a single output stream, so the ability to tell stdout apart from stderr is not possible when pty=True. As such, all output will appear on out_stream (see below) and be captured into the stdout result attribute. err_stream and stderr will always be empty when pty=True.

• replace_env (bool) – When True, causes the subprocess to receive the dictionary given to env as its entire shell environment, instead of updating a copy of os.environ (which is the default behavior). Default: False.

• shell (str) – Which shell binary to use. Default: /bin/bash (on Unix; COMSPEC or cmd.exe on Windows.)

• timeout –

  Cause the runner to submit an interrupt to the subprocess and raise CommandTimedOut, if the command takes longer than timeout seconds to execute. Defaults to None, meaning no timeout.

  在 1.3 版本加入.

• warn (bool) –

  Whether to warn and continue, instead of raising UnexpectedExit, when the executed command exits with a nonzero status. Default: False.

  备注

  This setting has no effect on exceptions, which will still be raised, typically bundled in ThreadException objects if they were raised by the IO worker threads.

  Similarly, WatcherError exceptions raised by StreamWatcher instances will also ignore this setting, and will usually be bundled inside Failure objects (in order to preserve the execution context).

  Ditto CommandTimedOut - basically, anything that prevents a command from actually getting to “exited with an exit code” ignores this flag.

• watchers –

  A list of StreamWatcher instances which will be used to scan the program’s stdout or stderr and may write into its stdin (typically bytes objects) in response to patterns or other heuristics.

  See 自动响应程序输出 for details on this functionality.

  Default: [].

返回:

Result, or a subclass thereof.

Raises:

UnexpectedExit, if the command exited nonzero and warn was False.

Raises:

Failure, if the command didn’t even exit cleanly, e.g. if a StreamWatcher raised WatcherError.

Raises:

ThreadException (if the background I/O threads encountered exceptions other than WatcherError).

在 1.0 版本加入.

make_promise() → Promise

Return a Promise allowing async control of the rest of lifecycle.

在 1.4 版本加入.

create_io_threads() → Tuple[Dict[Callable, ExceptionHandlingThread], List[str], List[str]]

Create and return a dictionary of IO thread worker objects.

Caller is expected to handle persisting and/or starting the wrapped threads.

generate_result(**kwargs: Any) → Result

Create & return a suitable Result instance from the given kwargs.

Subclasses may wish to override this in order to manipulate things or generate a Result subclass (e.g. ones containing additional metadata besides the default).

在 1.0 版本加入.

read_proc_output(reader: Callable) → Generator[str, None, None]

Iteratively read & decode bytes from a subprocess’ out/err stream.

参数:

reader –

A literal reader function/partial, wrapping the actual stream object in question, which takes a number of bytes to read, and returns that many bytes (or None).

reader should be a reference to either read_proc_stdout or read_proc_stderr, which perform the actual, platform/library specific read calls.

返回:

A generator yielding strings.

Specifically, each resulting string is the result of decoding read_chunk_size bytes read from the subprocess’ out/err stream.

在 1.0 版本加入.

write_our_output(stream: IO, string: str) → None

Write string to stream.

Also calls .flush() on stream to ensure that real terminal streams don’t buffer.

参数:

• stream – A file-like stream object, mapping to the out_stream or err_stream parameters of run.

• string – A Unicode string object.

返回:

None.

在 1.0 版本加入.

handle_stdout(buffer_: List[str], hide: bool, output: IO) → None

Read process’ stdout, storing into a buffer & printing/parsing.

Intended for use as a thread target. Only terminates when all stdout from the subprocess has been read.

参数:

• buffer – The capture buffer shared with the main thread.

• hide (bool) – Whether or not to replay data into output.

• output – Output stream (file-like object) to write data into when not hiding.

返回:

None.

在 1.0 版本加入.

handle_stderr(buffer_: List[str], hide: bool, output: IO) → None

Read process’ stderr, storing into a buffer & printing/parsing.

Identical to handle_stdout except for the stream read from; see its docstring for API details.

在 1.0 版本加入.

read_our_stdin(input_: IO) → str | None

Read & decode bytes from a local stdin stream.

参数:

input – Actual stream object to read from. Maps to in_stream in run, so will often be sys.stdin, but might be any stream-like object.

返回:

A Unicode string, the result of decoding the read bytes (this might be the empty string if the pipe has closed/reached EOF); or None if stdin wasn’t ready for reading yet.

在 1.0 版本加入.

handle_stdin(input_: IO, output: IO, echo: bool = False) → None

Read local stdin, copying into process’ stdin as necessary.

Intended for use as a thread target.

备注

Because real terminal stdin streams have no well-defined “end”, if such a stream is detected (based on existence of a callable .fileno()) this method will wait until program_finished is set, before terminating.

When the stream doesn’t appear to be from a terminal, the same semantics as handle_stdout are used - the stream is simply read() from until it returns an empty value.

参数:

• input – Stream (file-like object) from which to read.

• output – Stream (file-like object) to which echoing may occur.

• echo (bool) – User override option for stdin-stdout echoing.

返回:

None.

在 1.0 版本加入.

should_echo_stdin(input_: IO, output: IO) → bool

Determine whether data read from input_ should echo to output.

Used by handle_stdin; tests attributes of input_ and output.

参数:

• input – Input stream (file-like object).

• output – Output stream (file-like object).

返回:

A bool.

在 1.0 版本加入.

respond(buffer_: List[str]) → None

Write to the program’s stdin in response to patterns in buffer_.

The patterns and responses are driven by the StreamWatcher instances from the watchers kwarg of run - see 自动响应程序输出 for a conceptual overview.

参数:

buffer – The capture buffer for this thread’s particular IO stream.

返回:

None.

在 1.0 版本加入.

generate_env(env: Dict[str, Any], replace_env: bool) → Dict[str, Any]

Return a suitable environment dict based on user input & behavior.

参数:

• env (dict) – Dict supplying overrides or full env, depending.

• replace_env (bool) – Whether env updates, or is used in place of, the value of os.environ.

返回:

A dictionary of shell environment vars.

在 1.0 版本加入.

should_use_pty(pty: bool, fallback: bool) → bool

Should execution attempt to use a pseudo-terminal?

参数:

• pty (bool) – Whether the user explicitly asked for a pty.

• fallback (bool) – Whether falling back to non-pty execution should be allowed, in situations where pty=True but a pty could not be allocated.

在 1.0 版本加入.

property has_dead_threads: bool

Detect whether any IO threads appear to have terminated unexpectedly.

Used during process-completion waiting (in wait) to ensure we don’t deadlock our child process if our IO processing threads have errored/died.

返回:

True if any threads appear to have terminated with an exception, False otherwise.

在 1.0 版本加入.

wait() → None

Block until the running command appears to have exited.

返回:

None.

在 1.0 版本加入.

write_proc_stdin(data: str) → None

Write encoded data to the running process’ stdin.

参数:

data – A Unicode string.

返回:

None.

在 1.0 版本加入.

decode(data: bytes) → str

Decode some data bytes, returning Unicode.

在 1.0 版本加入.

property process_is_finished: bool

Determine whether our subprocess has terminated.

备注

The implementation of this method should be nonblocking, as it is used within a query/poll loop.

返回:

True if the subprocess has finished running, False otherwise.

在 1.0 版本加入.

start(command: str, shell: str, env: Dict[str, Any]) → None

Initiate execution of command (via shell, with env).

Typically this means use of a forked subprocess or requesting start of execution on a remote system.

In most cases, this method will also set subclass-specific member variables used in other methods such as wait and/or returncode.

参数:

• command (str) – Command string to execute.

• shell (str) – Shell to use when executing command.

• env (dict) – Environment dict used to prep shell environment.

在 1.0 版本加入.

start_timer(timeout: int) → None

Start a timer to kill our subprocess after timeout seconds.

read_proc_stdout(num_bytes: int) → bytes | None

Read num_bytes from the running process’ stdout stream.

参数:

num_bytes (int) – Number of bytes to read at maximum.

返回:

A string/bytes object.

在 1.0 版本加入.

read_proc_stderr(num_bytes: int) → bytes | None

Read num_bytes from the running process’ stderr stream.

参数:

num_bytes (int) – Number of bytes to read at maximum.

返回:

A string/bytes object.

在 1.0 版本加入.

close_proc_stdin() → None

Close running process’ stdin.

返回:

None.

在 1.3 版本加入.

default_encoding() → str

Return a string naming the expected encoding of subprocess streams.

This return value should be suitable for use by encode/decode methods.

在 1.0 版本加入.

send_interrupt(interrupt: KeyboardInterrupt) → None

Submit an interrupt signal to the running subprocess.

In almost all implementations, the default behavior is what will be desired: submit  to the subprocess’ stdin pipe. However, we leave this as a public method in case this default needs to be augmented or replaced.

参数:

interrupt – The locally-sourced KeyboardInterrupt causing the method call.

返回:

None.

在 1.0 版本加入.

returncode() → int | None

Return the numeric return/exit code resulting from command execution.

返回:

int, if any reasonable return code could be determined, or None in corner cases where that was not possible.

在 1.0 版本加入.

stop() → None

Perform final cleanup, if necessary.

This method is called within a finally clause inside the main run method. Depending on the subclass, it may be a no-op, or it may do things such as close network connections or open files.

返回:

None

在 1.0 版本加入.

kill() → None

Forcibly terminate the subprocess.

Typically only used by the timeout functionality.

This is often a “best-effort” attempt, e.g. remote subprocesses often must settle for simply shutting down the local side of the network connection and hoping the remote end eventually gets the message.

property timed_out: bool

Returns True if the subprocess stopped because it timed out.

在 1.3 版本加入.

__weakref__

list of weak references to the object

class invoke.runners.Local(context: Context)

Execute a command on the local system in a subprocess.

备注

When Invoke itself is executed without a controlling terminal (e.g. when sys.stdin lacks a useful fileno), it’s not possible to present a handle on our PTY to local subprocesses. In such situations, Local will fallback to behaving as if pty=False (on the theory that degraded execution is better than none at all) as well as printing a warning to stderr.

To disable this behavior, say fallback=False.

在 1.0 版本加入.

__init__(context: Context) → None

Create a new runner with a handle on some Context.

参数:

context –

a Context instance, used to transmit default options and provide access to other contextualized information (e.g. a remote-oriented Runner might want a Context subclass holding info about hostnames and ports.)

备注

The Context given to Runner instances must contain default config values for the Runner class in question. At a minimum, this means values for each of the default Runner.run keyword arguments such as echo and warn.

抛出:

exceptions.ValueError – if not all expected default values are found in context.

should_use_pty(pty: bool = False, fallback: bool = True) → bool

Should execution attempt to use a pseudo-terminal?

参数:

• pty (bool) – Whether the user explicitly asked for a pty.

• fallback (bool) – Whether falling back to non-pty execution should be allowed, in situations where pty=True but a pty could not be allocated.

在 1.0 版本加入.

read_proc_stdout(num_bytes: int) → bytes | None

Read num_bytes from the running process’ stdout stream.

参数:

num_bytes (int) – Number of bytes to read at maximum.

返回:

A string/bytes object.

在 1.0 版本加入.

read_proc_stderr(num_bytes: int) → bytes | None

Read num_bytes from the running process’ stderr stream.

参数:

num_bytes (int) – Number of bytes to read at maximum.

返回:

A string/bytes object.

在 1.0 版本加入.

close_proc_stdin() → None

Close running process’ stdin.

返回:

None.

在 1.3 版本加入.

start(command: str, shell: str, env: Dict[str, Any]) → None

Initiate execution of command (via shell, with env).

Typically this means use of a forked subprocess or requesting start of execution on a remote system.

In most cases, this method will also set subclass-specific member variables used in other methods such as wait and/or returncode.

参数:

• command (str) – Command string to execute.

• shell (str) – Shell to use when executing command.

• env (dict) – Environment dict used to prep shell environment.

在 1.0 版本加入.

kill() → None

Forcibly terminate the subprocess.

Typically only used by the timeout functionality.

This is often a “best-effort” attempt, e.g. remote subprocesses often must settle for simply shutting down the local side of the network connection and hoping the remote end eventually gets the message.

property process_is_finished: bool

Determine whether our subprocess has terminated.

备注

The implementation of this method should be nonblocking, as it is used within a query/poll loop.

返回:

True if the subprocess has finished running, False otherwise.

在 1.0 版本加入.

returncode() → int | None

Return the numeric return/exit code resulting from command execution.

返回:

int, if any reasonable return code could be determined, or None in corner cases where that was not possible.

在 1.0 版本加入.

stop() → None

Perform final cleanup, if necessary.

This method is called within a finally clause inside the main run method. Depending on the subclass, it may be a no-op, or it may do things such as close network connections or open files.

返回:

None

在 1.0 版本加入.

class invoke.runners.Result(stdout: str = '', stderr: str = '', encoding: str | None = None, command: str = '', shell: str = '', env: Dict[str, Any] | None = None, exited: int = 0, pty: bool = False, hide: Tuple[str, ...] = ())

A container for information about the result of a command execution.

All params are exposed as attributes of the same name and type.

参数:

• stdout (str) – The subprocess’ standard output.

• stderr (str) – Same as stdout but containing standard error (unless the process was invoked via a pty, in which case it will be empty; see Runner.run.)

• encoding (str) – The string encoding used by the local shell environment.

• command (str) – The command which was executed.

• shell (str) – The shell binary used for execution.

• env (dict) – The shell environment used for execution. (Default is the empty dict, {}, not None as displayed in the signature.)

• exited (int) –

  An integer representing the subprocess’ exit/return code.

  备注

  This may be None in situations where the subprocess did not run to completion, such as when auto-responding failed or a timeout was reached.

• pty (bool) – A boolean describing whether the subprocess was invoked with a pty or not; see Runner.run.

• hide (tuple) –

  A tuple of stream names (none, one or both of ('stdout', 'stderr')) which were hidden from the user when the generating command executed; this is a normalized value derived from the hide parameter of Runner.run.

  For example, run('command', hide='stdout') will yield a Result where result.hide == ('stdout',); hide=True or hide='both' results in result.hide == ('stdout', 'stderr'); and hide=False (the default) generates result.hide == () (the empty tuple.)

备注

Result objects’ truth evaluation is equivalent to their ok attribute’s value. Therefore, quick-and-dirty expressions like the following are possible:

if run("some shell command"):
    do_something()
else:
    handle_problem()

However, remember Zen of Python #2.

在 1.0 版本加入.

__init__(stdout: str = '', stderr: str = '', encoding: str | None = None, command: str = '', shell: str = '', env: Dict[str, Any] | None = None, exited: int = 0, pty: bool = False, hide: Tuple[str, ...] = ())

property return_code: int

An alias for .exited.

在 1.0 版本加入.

__str__() → str

Return str(self).

__repr__() → str

Return repr(self).

property ok: bool

A boolean equivalent to exited == 0.

在 1.0 版本加入.

property failed: bool

The inverse of ok.

I.e., True if the program exited with a nonzero return code, and False otherwise.

在 1.0 版本加入.

tail(stream: str, count: int = 10) → str

Return the last count lines of stream, plus leading whitespace.

参数:

• stream (str) – Name of some captured stream attribute, eg "stdout".

• count (int) – Number of lines to preserve.

在 1.3 版本加入.

__weakref__

list of weak references to the object

class invoke.runners.Promise(runner: Runner)

A promise of some future Result, yielded from asynchronous execution.

This class’ primary API member is join; instances may also be used as context managers, which will automatically call join when the block exits. In such cases, the context manager yields self.

Promise also exposes copies of many Result attributes, specifically those that derive from run kwargs and not the result of command execution. For example, command is replicated here, but stdout is not.

在 1.4 版本加入.

__init__(runner: Runner) → None

Create a new promise.

参数:

runner –

An in-flight Runner instance making this promise.

Must already have started the subprocess and spun up IO threads.

join() → Result

Block until associated subprocess exits, returning/raising the result.

This acts identically to the end of a synchronously executed run, namely that:

• various background threads (such as IO workers) are themselves joined;

• if the subprocess exited normally, a Result is returned;

• in any other case (unforeseen exceptions, IO sub-thread ThreadException, Failure, WatcherError) the relevant exception is raised here.

See run docs, or those of the relevant classes, for further details.

invoke.runners.default_encoding() → str

Obtain apparent interpreter-local default text encoding.

Often used as a baseline in situations where we must use SOME encoding for unknown-but-presumably-text bytes, and the user has not specified an override.