Google BigQuery

Frends.GoogleBigQuery.CreateResource / 2.0.0

Summary

Creates a BigQuery object — a dataset, table or similar — including its description, labels and optional customer-managed encryption, rather than only its name.

Typical use — provisioning the target a load Process will write into, so the schema and encryption policy live in the Process rather than in manual setup

Details

Frends version
5.7 and newer
Agent
Crossplatform
Target framework
net8.0

Input parameters.

Name Type Default Example Description
Resource Resources Dataset Resources.Dataset

Resource type.

Possible values:

  • Dataset: The dataset resource.
  • Routine: The routine resource.
  • Table: The table resource.
Description String – Example description.

Resource description. (Optional)

FriendlyName String – Example name

Friendly name for Dataset or Table resource. (Optional)

Label LabelParameters[] – [ {foo, bar }, { bar, foo } ]

Label(s) for Dataset or Table resource. (Optional) Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label keys must start with a letter and each label in the list must have a different key.

SetEncryptionConfiguration Boolean False false

Set Dataset or Table encryption configuration.

KmsKeyName String – projects/your-project/locations/global/keyRings/your-key-ring/cryptoKeys/your-key

Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

DatasetId String – Exampleid

A unique ID for this dataset, without the project name. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.

Location String EU europe-north1

The geographic location where the dataset should reside. See details at https://cloud.google.com/bigquery/docs/locations

StorageBillingModel String LOGICAL LOGICAL

Storage billing model to be used for all tables in the dataset. Can be set to PHYSICAL.

IsCaseInsensitive Boolean False false

Indicates if table names are case insensitive in the dataset.

MaxTimeTravelHours Int64 0 0

Number of hours for the max time travel for all tables in the dataset. Unlimited if 0.

Access AccessParameters[] – [ { READER, foo@bar.com }, { WRITER, bar@bar.com } ]

An array of objects that define dataset access for one or more entities. (Optional) You can set this property when inserting or updating a dataset in order to control who is allowed to access the data. If unspecified at dataset creation time, BigQuery adds default dataset access for the following entities: access.specialGroup: projectReaders; access.role: READER; access.specialGroup: projectWriters; access.role: WRITER; access.specialGroup: projectOwners; access.role: OWNER; access.userByEmail: [dataset creator email]; access.role: OWNER;

Tag TagParameters[] – [ { foo, bar }, { foo2, bar2 } ]

The tags associated with this dataset. (Optional) Tag keys are globally unique.

RoutineId String – routine

The ID of the routine. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 256.

DefinitionBody String – concat(x, \"\\n\", y)

The body of the routine. For functions, this is the expression in the AS clause. If language=SQL, it is the substring inside (but excluding) the parentheses. For example, for the function created with the following statement: CREATE FUNCTION JoinLines(x string, y string) as (concat(x, "\n", y)) The definition_body is concat(x, "\n", y) (\n is not replaced with linebreak). If language=JAVASCRIPT, it is the evaluated string in the AS clause. For example, for the function created with the following statement: CREATE FUNCTION f() RETURNS STRING LANGUAGE js AS 'return "\n";\n' The definition_body is return "\n";\n Note that both \n are replaced with linebreaks.

Language String – SQL

Defaults to "SQL" if Input.RemoteFunctionParameters is not empty, not set otherwise. (Optional)

RoutineType RoutineTypes SCALAR_FUNCTION RoutineTypes.SCALAR_FUNCTION

Routine type.

Possible values:

  • SCALAR_FUNCTION: A scalar function.
  • PROCEDURE: A stored procedure.
  • TABLE_FUNCTION: A table function.
ReturnType String – DATE

Optional if language = "SQL"; required otherwise. Cannot be set if Input.RoutineType = "TABLE_VALUED_FUNCTION". If absent, the return type is inferred from definition_body at query time in each query that references this routine. If present, then the evaluated result will be cast to the specified returned type at query time. For example, for the functions created with the following statements: * CREATE FUNCTION Add(x FLOAT64, y FLOAT64) RETURNS FLOAT64 AS (x + y); * CREATE FUNCTION Increment(x FLOAT64) AS (Add(x, 1)); * CREATE FUNCTION Decrement(x FLOAT64) RETURNS FLOAT64 AS (Add(x, -1)); The return_type is {type_kind: "FLOAT64"} for Add and Decrement, and is absent for Increment (inferred as FLOAT64 at query time). Suppose the function Add is replaced by CREATE OR REPLACE FUNCTION Add(x INT64, y INT64) AS (x + y); Then the inferred return type of Increment is automatically changed to INT64 at query time, while the return type of Decrement remains FLOAT64.

DeterminismLevel DeterminismLevels DETERMINISM_LEVEL_UNSPECIFIED DeterminismLevels.DETERMINISTIC

The determinism level of the JavaScript UDF, if defined.

Possible values:

  • DETERMINISM_LEVEL_UNSPECIFIED: The determinism of the UDF is unspecified.

  • DETERMINISTIC: The UDF is deterministic, meaning that 2 function calls with the same inputs always produce the same result, even across 2 query runs.

  • NOT_DETERMINISTIC: The UDF is not deterministic.

StrictMode Boolean True true

Can be set for procedures only. If true (default), the definition body will be validated in the creation and the updates of the procedure. For procedures with an argument of ANY TYPE, the definition body validtion is not supported at creation/update time, and thus this field must be set to false explicitly.

ImportedLibrary ImportedLibraryParameters[] – [ { foo }, { bar } ]

If Input.Language = "JAVASCRIPT", this field stores the path of the imported JAVASCRIPT libraries. (Optional)

Argument ArgumentParameters[] – [ { x, IN, INT64, FIXED_TYPE }, { y, OUT, INT64, FIXED_TYPE }, { 'not set', 'not set', INT64, 'not set' } ]

Argument(s) of a function or a stored procedure.

SetRemoteFunctionParameters Boolean False false

Set remote function specific options.

RemoteConnection String – projects/{projectId}/locations/{locationId}/connections/{connectionId}

Fully qualified name of the user-provided connection object which holds the authentication information to send requests to the remote service.

RemoteEndpoint String – https://us-east1-my_gcf_project.cloudfunctions.net/remote_add

Endpoint of the user-provided remote service.

RemoteMaxBatchingRows Int64 – 0

Max number of rows in each batch sent to the remote service. If 0, BigQuery dynamically decides the number of rows in a batch.

RemoteUserDefinedContext UserDefinedContext[] – [ { foo, bar }, { foo2, bar2 } ]

User-defined context as a set of key/value pairs, which will be sent as function invocation context together with batched arguments in the requests to the remote service. The total number of bytes of keys and values must be less than 8KB.

TableId String – tableid

Table ID.

RequirePartitionFilter Boolean False false

If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.

TableSchema TableSchemaParameters[] – [ { Name = "Name", Type = "STRING" }, { Name = "AGE", Type = "INTEGER" }, { Name = "BDAY", Type = "DATE" } ]

Describes the schema of this table.

Partition TablePartitionOptions None TablePartitionOptions.RangePartitioning

Partition option.

Possible values:

  • None: Hide options from UI.

  • RangePartitioning: Range partitioning specification for this table.

  • TimePartitioning: Time-based partitioning specification for this table.

RangeField String – AGE

The table is partitioned by this field. The field must be a top-level NULLABLE/REQUIRED field. The only supported type is INTEGER/INT64.

RangeStart Int64 – 1

The start of range partitioning, inclusive

RangeEnd Int64 – 2

The end of range partitioning, exclusive.

RangeInterval Int64 – 2

The width of each interval.

TimeField String – TIMESTAMP

If not set, the table is partitioned by pseudo column, referenced via either '_PARTITIONTIME' as TIMESTAMP type, or '_PARTITIONDATE' as DATE type. If field is specified, the table is instead partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED.

Type TimeTypeOptions DAY TimeTypeOptions.DAY

Generate one partition per day, hour, month, or year.

Possible values:

  • DAY: Partition by day.
  • HOUR: Partition by hour.
  • MONTH: Partition by month.
  • YEAR: Partition by year.
TimeRequirePartitionFilter Boolean false false

RequirePartitionFilter

ExpirationMs Int64 0 0

Number of milliseconds for which to keep the storage for partitions in the table. The storage in a partition will have an expiration time of its partition time plus this value.

An unhandled error has occurred. Reload 🗙