jina gateway

Start a Gateway that receives client Requests via gRPC/REST interface

usage: jina gateway [-h] [--name] [--description] [--log-config] [--quiet]
                    [--quiet-error] [--port-ctrl] [--ctrl-with-ipc]
                    [--timeout-ctrl] [--ssh-server] [--ssh-keyfile]
                    [--ssh-password] [--uses] [--py-modules [PATH [PATH ...]]]
                    [--port-in] [--port-out] [--host-in] [--host-out]
                    [--socket-in {PULL_BIND, PULL_CONNECT, PUSH_BIND, PUSH_CONNECT ... 8 more choices}]
                    [--socket-out {PULL_BIND, PULL_CONNECT, PUSH_BIND, PUSH_CONNECT ... 8 more choices}]
                    [--load-interval] [--dump-interval] [--read-only]
                    [--memory-hwm]
                    [--on-error-strategy {IGNORE, SKIP_EXECUTOR, SKIP_HANDLE, THROW_EARLY}]
                    [--max-message-size] [--proxy] [--prefetch]
                    [--prefetch-on-recv] [--restful]
                    [--compress {NONE, LZ4, ZLIB, GZIP ... 2 more choices}]
                    [--compress-min-bytes] [--compress-min-ratio] [--host]
                    [--port-expose] [--daemon]
                    [--runtime-backend {THREAD, PROCESS}]
                    [--runtime-cls {GRPCRuntime, RESTRuntime, ContainerRuntime, JinadRuntime ... 2 more choices}]
                    [--timeout-ready] [--env [KEY=VALUE [KEY=VALUE ...]]]
                    [--expose-public]

Essential arguments

--name

The name of this object.

This will be used in the following places: - how you refer to this object in Python/YAML/CLI - visualization - log message header - automatics docs UI - …

When not given, then the default naming strategy will apply.

Default: “gateway”

--description

The description of this object. It will be used in automatics docs UI.

--log-config

The YAML config of the logger used in this object.

Default: “/opt/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/jina/resources/logging.default.yml”

--quiet

If set, then no log will be emitted from this object.

Default: False

--quiet-error

If set, then exception stack information will not be added to the log

Default: False

ZMQRuntime arguments

--port-ctrl

The port for controlling the runtime, default a random port between [49152, 65535]

Default: 59029

--ctrl-with-ipc

If set, use ipc protocol for control socket

Default: True

--timeout-ctrl

The timeout in milliseconds of the control request, -1 for waiting forever

Default: 5000

--ssh-server

The SSH server through which the tunnel will be created, can actually be a fully specified user@server:port ssh url.

--ssh-keyfile

This specifies a key to be used in ssh login, default None. regular default ssh keys will be used without specifying this argument.

--ssh-password

The ssh password to the ssh server.

ZEDRuntime arguments

--uses

The config of the executor, it could be one of the followings: - an Executor-level YAML file path (.yml, .yaml, .jaml) - a name of a class inherited from jina.Executor - a docker image (must start with docker://) - builtin executors, e.g. _pass, _logforward, _merge - the string literal of a YAML config (must start with !) - the string literal of a JSON config - the string literal of a YAML driver config (must start with - !!)

When use it under Python, one can use the following values additionally: - a Python dict that represents the config - a text file stream has .read() interface

Default: “_pass”

--py-modules

The customized python modules need to be imported before loading the executor

Note, when importing multiple files and there is a dependency between them, then one has to write the dependencies in reverse order. That is, if __init__.py depends on A.py, which again depends on B.py, then you need to write:

–py-modules __init__.py –py-modules B.py –py-modules A.py

--port-in

The port for input data, default a random port between [49152, 65535]

Default: 49779

--port-out

The port for output data, default a random port between [49152, 65535]

Default: 36221

--host-in

The host address for input, by default it is 0.0.0.0

Default: “0.0.0.0”

--host-out

The host address for output, by default it is 0.0.0.0

Default: “0.0.0.0”

--socket-in

Possible choices: PULL_BIND, PULL_CONNECT, PUSH_BIND, PUSH_CONNECT, SUB_BIND, SUB_CONNECT, PUB_BIND, PUB_CONNECT, PAIR_BIND, PAIR_CONNECT, ROUTER_BIND, DEALER_CONNECT

The socket type for input port

Default: PULL_CONNECT

--socket-out

Possible choices: PULL_BIND, PULL_CONNECT, PUSH_BIND, PUSH_CONNECT, SUB_BIND, SUB_CONNECT, PUB_BIND, PUB_CONNECT, PAIR_BIND, PAIR_CONNECT, ROUTER_BIND, DEALER_CONNECT

The socket type for output port

Default: PUSH_CONNECT

--load-interval

Reload the Executor in the Pod on every n seconds. -1 or 0 means do not reload.

Default: -1

--dump-interval

Serialize the Executor in the Pod every n seconds if model changes. -1 means –read-only.

Default: 240

--read-only

If set, do not allow the pod to modify the model, dump_interval will be ignored

Default: True

--memory-hwm

The memory high watermark of this pod in Gigabytes, pod will restart when this is reached. -1 means no restriction

Default: -1

--on-error-strategy

Possible choices: IGNORE, SKIP_EXECUTOR, SKIP_HANDLE, THROW_EARLY

The skip strategy on exceptions.

  • IGNORE: Ignore it, keep running all Drivers & Executors logics in the sequel flow

  • SKIP_EXECUTOR: Skip all Executors in the sequel, but drivers are still called

  • SKIP_HANDLE: Skip all Drivers & Executors in the sequel, only pre_hook and post_hook are called

  • THROW_EARLY: Immediately throw the exception, the sequel flow will not be running at all

Note, IGNORE, SKIP_EXECUTOR and SKIP_HANDLE do not guarantee the success execution in the sequel flow. If something is wrong in the upstream, it is hard to carry this exception and moving forward without any side-effect.

Default: IGNORE

GRPC/REST arguments

--max-message-size

The maximum send and receive size for gRPC server in bytes, -1 means unlimited

Default: -1

--proxy

If set, respect the http_proxy and https_proxy environment variables. otherwise, it will unset these proxy variables before start. gRPC seems to prefer no proxy

Default: False

--prefetch

The number of pre-fetched requests from the client

Default: 50

--prefetch-on-recv

The number of additional requests to fetch on every receive

Default: 1

--restful, --rest-api

If set, use RESTful interface instead of gRPC as the main interface

Default: False

--compress

Possible choices: NONE, LZ4, ZLIB, GZIP, BZ2, LZMA

The compress algorithm used over the entire Flow.

Note that this is not necessarily effective, it depends on the settings of –compress-lwm and compress-hwm

Default: LZ4

--compress-min-bytes

The original message size must be larger than this number to trigger the compress algorithm, -1 means disable compression.

Default: 1024

--compress-min-ratio

The compression ratio (uncompressed_size/compressed_size) must be higher than this number to trigger the compress algorithm.

Default: 1.1

Expose arguments

--host

The host address of the runtime, by default it is 0.0.0.0.

Default: “0.0.0.0”

--port-expose

The port of the host exposed to the public

Default: 48813

Pea arguments

--daemon

The Pea attempts to terminate all of its Runtime child processes/threads on existing. setting it to true basically tell the Pea do not wait on the Runtime when closing

Default: False

--runtime-backend, --runtime

Possible choices: THREAD, PROCESS

The parallel backend of the runtime inside the Pea

Default: PROCESS

--runtime-cls

Possible choices: GRPCRuntime, RESTRuntime, ContainerRuntime, JinadRuntime, SSHRuntime, ZEDRuntime

The runtime class to run inside the Pea

Default: “GRPCRuntime”

--timeout-ready

The timeout in milliseconds of a Pea waits for the runtime to be ready, -1 for waiting forever

Default: 60000

--env

The map of environment variables that are available inside runtime

--expose-public

If set, expose the public IP address to remote when necessary, by default it exposesprivate IP address, which only allows accessing under the same network/subnet. Important to set this to true when the Pea will receive input connections from remote Peas

Default: False