Dataflow API . projects

Instance Methods

jobs()

Returns the jobs Resource.

locations()

Returns the locations Resource.

snapshots()

Returns the snapshots Resource.

templates()

Returns the templates Resource.

deleteSnapshots(projectId, location=None, snapshotId=None, x__xgafv=None)

Deletes a snapshot.

workerMessages(projectId, body, x__xgafv=None)

Send a worker_message to the service.

Method Details

deleteSnapshots(projectId, location=None, snapshotId=None, x__xgafv=None)
Deletes a snapshot.

Args:
  projectId: string, The ID of the Cloud Platform project that the snapshot belongs to. (required)
  location: string, The location that contains this snapshot.
  snapshotId: string, The ID of the snapshot.
  x__xgafv: string, V1 error format.
    Allowed values
      1 - v1 error format
      2 - v2 error format

Returns:
  An object of the form:

    { # Response from deleting a snapshot.
  }
workerMessages(projectId, body, x__xgafv=None)
Send a worker_message to the service.

Args:
  projectId: string, The project to send the WorkerMessages to. (required)
  body: object, The request body. (required)
    The object takes the form of:

{ # A request for sending worker messages to the service.
    "workerMessages": [ # The WorkerMessages to send.
      { # WorkerMessage provides information to the backend about a worker.
        "workerShutdownNotice": { # Shutdown notification from workers. This is to be sent by the shutdown # Shutdown notice by workers.
            # script of the worker VM so that the backend knows that the VM is being
            # shut down.
          "reason": "A String", # The reason for the worker shutdown.
              # Current possible values are:
              #   "UNKNOWN": shutdown reason is unknown.
              #   "PREEMPTION": shutdown reason is preemption.
              # Other possible reasons may be added in the future.
        },
        "workerMetrics": { # Worker metrics exported from workers. This contains resource utilization # Resource metrics reported by workers.
            # metrics accumulated from a variety of sources. For more information, see
            # go/df-resource-signals.
          "cpuTime": [ # CPU utilization samples.
            { # Modeled after information exposed by /proc/stat.
              "totalMs": "A String", # Total active CPU time across all cores (ie., non-idle) in milliseconds
                  # since start-up.
              "rate": 3.14, # Average CPU utilization rate (% non-idle cpu / second) since previous
                  # sample.
              "timestamp": "A String", # Timestamp of the measurement.
            },
          ],
        },
        "workerMessageCode": { # A message code is used to report status and error messages to the service. # A worker message code.
            # The message codes are intended to be machine readable. The service will
            # take care of translating these into user understandable messages if
            # necessary.
            #
            # Example use cases:
            #   1. Worker processes reporting successful startup.
            #   2. Worker processes reporting specific errors (e.g. package staging
            #      failure).
          "code": "A String", # The code is a string intended for consumption by a machine that identifies
              # the type of message being sent.
              # Examples:
              #  1. "HARNESS_STARTED" might be used to indicate the worker harness has
              #      started.
              #  2. "GCS_DOWNLOAD_ERROR" might be used to indicate an error downloading
              #     a GCS file as part of the boot process of one of the worker containers.
              #
              # This is a string and not an enum to make it easy to add new codes without
              # waiting for an API change.
          "parameters": { # Parameters contains specific information about the code.
              #
              # This is a struct to allow parameters of different types.
              #
              # Examples:
              #  1. For a "HARNESS_STARTED" message parameters might provide the name
              #     of the worker and additional data like timing information.
              #  2. For a "GCS_DOWNLOAD_ERROR" parameters might contain fields listing
              #     the GCS objects being downloaded and fields containing errors.
              #
              # In general complex data structures should be avoided. If a worker
              # needs to send a specific and complicated data structure then please
              # consider defining a new proto and adding it to the data oneof in
              # WorkerMessageResponse.
              #
              # Conventions:
              #  Parameters should only be used for information that isn't typically passed
              #  as a label.
              #  hostname and other worker identifiers should almost always be passed
              #  as labels since they will be included on most messages.
            "a_key": "", # Properties of the object.
          },
        },
        "labels": { # Labels are used to group WorkerMessages.
            # For example, a worker_message about a particular container
            # might have the labels:
            # { "JOB_ID": "2015-04-22",
            #   "WORKER_ID": "wordcount-vm-2015…"
            #   "CONTAINER_TYPE": "worker",
            #   "CONTAINER_ID": "ac1234def"}
            # Label tags typically correspond to Label enum values. However, for ease
            # of development other strings can be used as tags. LABEL_UNSPECIFIED should
            # not be used here.
          "a_key": "A String",
        },
        "workerLifecycleEvent": { # A report of an event in a worker's lifecycle. # Record of worker lifecycle events.
            # The proto contains one event, because the worker is expected to
            # asynchronously send each message immediately after the event.
            # Due to this asynchrony, messages may arrive out of order (or missing), and it
            # is up to the consumer to interpret.
            # The timestamp of the event is in the enclosing WorkerMessage proto.
          "event": "A String", # The event being reported.
          "containerStartTime": "A String", # The start time of this container. All events will report this so that
              # events can be grouped together across container/VM restarts.
          "metadata": { # Other stats that can accompany an event. E.g.
              # { "downloaded_bytes" : "123456" }
            "a_key": "A String",
          },
        },
        "time": "A String", # The timestamp of the worker_message.
        "workerHealthReport": { # WorkerHealthReport contains information about the health of a worker. # The health of a worker.
            #
            # The VM should be identified by the labels attached to the WorkerMessage that
            # this health ping belongs to.
          "vmIsBroken": True or False, # Whether the VM is in a permanently broken state.
              # Broken VMs should be abandoned or deleted ASAP to avoid assigning or
              # completing any work.
          "vmStartupTime": "A String", # The time the VM was booted.
          "vmIsHealthy": True or False, # Whether the VM is currently healthy.
          "msg": "A String", # A message describing any unusual health reports.
          "reportInterval": "A String", # The interval at which the worker is sending health reports.
              # The default value of 0 should be interpreted as the field is not being
              # explicitly set by the worker.
          "pods": [ # The pods running on the worker. See:
              # http://kubernetes.io/v1.1/docs/api-reference/v1/definitions.html#_v1_pod
              #
              # This field is used by the worker to send the status of the indvidual
              # containers running on each worker.
            {
              "a_key": "", # Properties of the object.
            },
          ],
        },
      },
    ],
    "location": "A String", # The [regional endpoint]
        # (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that
        # contains the job.
  }

  x__xgafv: string, V1 error format.
    Allowed values
      1 - v1 error format
      2 - v2 error format

Returns:
  An object of the form:

    { # The response to the worker messages.
    "workerMessageResponses": [ # The servers response to the worker messages.
      { # A worker_message response allows the server to pass information to the
          # sender.
        "workerHealthReportResponse": { # WorkerHealthReportResponse contains information returned to the worker # The service's response to a worker's health report.
            # in response to a health ping.
          "reportInterval": "A String", # A positive value indicates the worker should change its reporting interval
              # to the specified value.
              #
              # The default value of zero means no change in report rate is requested by
              # the server.
        },
        "workerMetricsResponse": { # Service-side response to WorkerMessage reporting resource utilization. # Service's response to reporting worker metrics (currently empty).
        },
        "workerShutdownNoticeResponse": { # Service-side response to WorkerMessage issuing shutdown notice. # Service's response to shutdown notice (currently empty).
        },
      },
    ],
  }