# Introduction to object tables Object tables are read-only tables over unstructured data objects in Cloud Storage. They let you analyze unstructured data by using remote functions or BigQuery ML inference, and then join the results with the rest of your structured data in BigQuery. Like BigLake tables, object tables use access delegation, so you grant users access to the object table instead of to the underlying Cloud Storage objects. This design lets you enforce row-level security and manage which objects users can access. > [!NOTE] > **Note:** When managing access for users in [external identity providers](https://docs.cloud.google.com/iam/docs/workforce-identity-federation), replace instances of Google Account principal identifiers---like `user:kiran@example.com`, `group:support@example.com`, and `domain:example.com`---with appropriate [Workforce Identity Federation principal identifiers](https://docs.cloud.google.com/iam/docs/principal-identifiers). ## Create object tables You can use the [`CREATE EXTERNAL TABLE` statement](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_external_table_statement) to create an object table, as shown in the following example: CREATE EXTERNAL TABLE `myproject.mydataset.myobjecttable` WITH CONNECTION `myproject.us.myconnection` OPTIONS ( object_metadata = 'SIMPLE', uris = ['gs://mybucket/*'] ); To learn more about creating object tables, see [Create object tables](https://docs.cloud.google.com/bigquery/docs/object-tables). ## Object table schema An object table provides a metadata index over the unstructured data objects in a specified Cloud Storage bucket. Each row of the table corresponds to an object, and the table columns correspond to the object metadata generated by Cloud Storage, including any [custom metadata](https://docs.cloud.google.com/storage/docs/metadata#custom-metadata). An object table also contains a `data` pseudocolumn that represents the file content in raw bytes, which is autopopulated when the object table is created. This pseudocolumn is used by the [`ML.DECODE_IMAGE` function](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-decode-image) when you run inference on image data. You can't include the `data` pseudocolumn in queries, and it doesn't appear as part of the object table schema. The following table describes the fixed schema used by object tables: | **Field name** | **Type** | **Mode** | **Description** | |---|---|---|---| | `uri` | STRING | NULLABLE | `uri`: the Uniform Resource Identifier (URI) of the object, in the format `gs://bucket_name/[folder_name/]object_name`. | | `generation` | INTEGER | NULLABLE | The [generation](https://docs.cloud.google.com/storage/docs/metadata#generation-number) of this object, which identifies the object version. | | `content_type` | STRING | NULLABLE | The [Content-Type](https://docs.cloud.google.com/storage/docs/metadata#content-type) of the object data, which identifies what kind of media it is. If an object is stored without a Content-Type, it is served as application/octet-stream. | | `size` | INTEGER | NULLABLE | The [Content-Length](https://datatracker.ietf.org/doc/html/rfc7230#section-3.3.2) of the data in bytes. | | `md5_hash` | STRING | NULLABLE | The [MD5 hash](https://wikipedia.org/wiki/MD5) of the data, encoded using [base64](https://datatracker.ietf.org/doc/html/rfc4648#section-4). For more information about using the MD5 hash, see [Cloud Storage object metadata](https://docs.cloud.google.com/storage/docs/metadata#md5). | | `updated` | TIMESTAMP | NULLABLE | The last time the object's metadata was modified. | | `metadata` | RECORD | REPEATED | [Custom metadata](https://docs.cloud.google.com/storage/docs/metadata#custom-metadata) for the object. Each piece of metadata is represented as a key-value pair in the child `(metadata.)name` and `(metadata.)value` fields of the `metadata` field. | | `(metadata.)name` | STRING | NULLABLE | Key in an individual metadata entry. | | `(metadata.)value` | STRING | NULLABLE | Value in an individual metadata entry. | | `ref` | STRUCT | NULLABLE | Google-managed Cloud Storage metadata stored in the [`ObjectRef` format](https://docs.cloud.google.com/bigquery/docs/work-with-objectref). You can use this column [to maintain `ObjectRef` values in standard tables](https://docs.cloud.google.com/bigquery/docs/objectref-columns). `ObjectRef` values let you integrate object data with structured data. | The rows in an object table look similar to the following: --- | uri | generation | content_type | size | md5_hash | updated | metadata...name | metadata...value | ref.uri | ref.version | ref.authorizer | ref.details | --- | gs://mybucket/a.jpeg | 165842... | image/jpeg | 26797 | 8c33be10f... | 2022-07-21 17:35:40.148000 UTC | null | null | gs://mybucket/a.jpeg | 12345678 | us.conn | {"gcs_metadata":{"content_type":"image/jpeg","md5_hash"... | --- | gs://mybucket/b.bmp | 305722... | image/bmp | 57932 | 44eb90cd1... | 2022-05-14 12:09:38.114000 UTC | null | null | gs://mybucket/b.bmp | 23456789 | us.conn | {"gcs_metadata":{"content_type":"image/bmp","md5_hash"... | --- ## Use cases You can query the metadata in an object table in the same way you would query any other BigQuery table. However, the primary use case for object tables is to make unstructured data accessible for analysis. You can use BigQuery ML to [run inference on image object tables](https://docs.cloud.google.com/bigquery/docs/object-table-inference) with TensorFlow, TensorFlow Lite, and PyTorch models. You can also use [remote functions](https://docs.cloud.google.com/bigquery/docs/remote-functions) to [analyze unstructured data](https://docs.cloud.google.com/bigquery/docs/object-table-remote-function) almost any way you want to. For example, you could create a remote function that lets you analyze images by using [Cloud Vision](https://docs.cloud.google.com/vision/docs), or one that lets you extract metadata from PDF documents by using [Apache Tika](https://tika.apache.org/). The following table describes the integration points you can use to do machine learning on object table data: | **Integration** | **Description** | **Use case** | **Tutorial** | |---|---|---|---| | The [`AI.GENERATE_TEXT` function](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-ai-generate-text) | Generate text by using a Gemini Enterprise Agent Platform, partner, or open model. | You want to generate text from object data. | [Generate text by using the `AI.GENERATE_TEXT` function](https://docs.cloud.google.com/bigquery/docs/generate-text) | | The [`AI.GENERATE_EMBEDDING` function](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-ai-generate-embedding) | Generate embeddings by using a Gemini Enterprise Agent Platform multimodal model. | You want to generate embeddings for video or image data to use in vector searches, model input, or other use cases. | [Generate image embeddings by using the `AI.GENERATE_EMBEDDING` function](https://docs.cloud.google.com/bigquery/docs/generate-visual-content-embedding) [Generate video embeddings by using the `AI.GENERATE_EMBEDDING` function](https://docs.cloud.google.com/bigquery/docs/generate-video-embedding) | | [Imported BigQuery ML models](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/inference-overview#inference_using_imported_models) | Import [TensorFlow](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-create-tensorflow), [TensorFlow Lite](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-create-tflite), or [ONNX](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-create-onnx) models to BigQuery ML to run local inference in BigQuery . | You are using open-source or custom models that fit within [supported limitations](https://docs.cloud.google.com/bigquery/docs/object-table-inference#limitations). | [Tutorial: Run inference on an object table by using a feature vector model](https://docs.cloud.google.com/bigquery/docs/inference-tutorial-mobilenet) | | [Cloud Run functions](https://docs.cloud.google.com/bigquery/docs/object-table-remote-function) | Use Cloud Run functions to call services or hosted models. This is the most generic integration. | You are self-hosting your models on Compute Engine, Google Kubernetes Engine, or other customer-owned infrastructure. | | | The [`ML.ANNOTATE_IMAGE` function](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-annotate-image) | Use the Cloud Vision API to annotate images. | You want to annotate images by using a Vision API pre-trained model. | [Annotate images with the `ML.ANNOTATE_IMAGE` function](https://docs.cloud.google.com/bigquery/docs/annotate-image) | | The [`ML.PROCESS_DOCUMENT` function](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-process-document) | Use the Document AI API to extract document insights. | You want to use Document AI pre-trained or custom document processors. | [Process documents with the `ML.PROCESS_DOCUMENT` function](https://docs.cloud.google.com/bigquery/docs/process-document) | | The [`ML.TRANSCRIBE` function](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-transcribe) | Use the Speech-to-Text API to transcribe audio files. | You want to use Speech-to-Text pre-trained or custom speech recognizers. | [Transcribe audio files with the `ML.TRANSCRIBE` function](https://docs.cloud.google.com/bigquery/docs/transcribe) | You can create a view or table from the results of your analysis if you want to join your results with other structured data. For example, the following statement creates a table based on inference results: ```googlesql CREATE TABLE my_dataset.my_inference_results AS SELECT uri, content_type, vision_feature FROM ML.PREDICT( MODEL my_dataset.vision_model, SELECT ML.DECODE_IMAGE(data) AS vision_input FROM my_dataset.object_table ); ``` After the table is created, you can join it with other tables based on either standard or custom metadata fields, as shown following: ```googlesql SELECT a.vision_feature, a.uri, b.description FROM my_dataset.my_inference_results a JOIN my_dataset.image_description b ON a.uri = b.uri; ``` You can also [create a search index](https://docs.cloud.google.com/bigquery/docs/search-index) to power searches over the results of your analysis. For example, the following statement creates a search index over data extracted from PDF files: ```googlesql CREATE SEARCH INDEX my_index ON pdf_text_extract(ALL COLUMNS); ``` You can then use the index to find what you need in those results: ```googlesql SELECT * FROM pdf_text_extract WHERE SEARCH(pdf_text, 'Google'); ``` ## Benefits Analyzing unstructured data natively in BigQuery provides the following benefits: - It reduces manual effort by letting you automate pre-processing steps such as tuning image sizes to model requirements. - It lets you use the familiar SQL interface to work with unstructured data. - It helps you save costs by utilizing existing BigQuery slots instead of having to provision new forms of compute. ## Signed URLs To get access to the data represented by an object, generate a signed URL. You can use the signed URL to directly view the object data, and you can also [pass signed URLs to remote functions](https://docs.cloud.google.com/bigquery/docs/object-table-remote-function) to enable them to work with object table data. Use the [`EXTERNAL_OBJECT_TRANSFORM` function](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/table-functions-built-in#external_object_transform) to generate signed URLs, as shown in the following example: ```googlesql SELECT uri, signed_url FROM EXTERNAL_OBJECT_TRANSFORM(TABLE `mydataset.myobjecttable`, ['SIGNED_URL']); ``` This returns results similar to the following: --- | uri | signed_url | --- | gs://mybucket/a.docx | https://storage.googleapis.com/mybucket/a.docx?X-Goog-Signature=abcd&... | --- | gs://mybucket/b.pdf | https://storage.googleapis.com/mybucket/b.pdf?X-Goog-Signature=wxyz&... | --- Signed URLs generated from object tables allow any user or procedure that possesses them to read the corresponding objects. Generated signed URLs expire after 6 hours. For more information, see [Cloud Storage Signed URLs](https://docs.cloud.google.com/storage/docs/access-control/signed-urls). ## Access control Object tables are built on top of BigLake, so they use an [external connection](https://docs.cloud.google.com/bigquery/docs/working-with-connections) based on a service account to access Cloud Storage data. This decouples access to the table from access to the underlying object store through access delegation. You grant the service account permissions to access data and metadata from the objects and surface it in the table. You grant users permissions only on the table, where you can govern data access by using Identity and Access Management (IAM) and [row-level security](https://docs.cloud.google.com/bigquery/docs/row-level-security-intro). Object tables vary from other tables that use access delegation, in that having access to a row of an object table confers access to the underlying file content. While a user can't access the object directly, they can generate a [signed URL](https://docs.cloud.google.com/bigquery/docs/object-table-introduction#signed_urls) that lets them see the file contents. For example, if the user has access to the object table row representing the `flower.jpg` image file, they can generate a signed URL to display the file and see that it is a picture of a daisy. Setting a row-level access policy on an object table restricts a user or group's access to the object metadata in selected rows, and also to the objects represented by those rows. For example, the following statement grants the user Alice access only to rows that represent objects created before June 25, 2022: ```googlesql CREATE ROW ACCESS POLICY before_20220625 ON my_dataset.my_object_table GRANT TO ("user:alice@example.com") FILTER USING (updated < TIMESTAMP("2022-06-25")); ``` With this row-level access policy in place, the following outcomes are true for Alice: - Running the query `SELECT * FROM my_dataset.my_object_table;` only returns rows that have an `updated` value prior to June 25, 2022. - Running inference on `my_dataset.my_object_table` only returns predictions for objects that have an `updated` value prior to June 25, 2022. - Generating signed URLs for `my_dataset.my_object_table` by using the [`EXTERNAL_OBJECT_TRANSFORM` function](https://docs.cloud.google.com/bigquery/docs/object-table-introduction#signed_urls) only creates URLs for objects that have an `updated` value prior to June 25, 2022. > [!WARNING] > **Warning:** Do not use the [`bigquery.objectRefs.read`](https://docs.cloud.google.com/bigquery/docs/access-control#bigquery.objectRefs.read) or [`bigquery.objectRefs.write`](https://docs.cloud.google.com/bigquery/docs/access-control#bigquery.objectRefs.write) permissions on the object table's connection with row-level access policies, because these permissions are designed to grant connection-level access to any object accessible to the connection's service account, bypassing row-level access policies. You can also restrict access to object table rows by using custom metadata. For example, the following statement restricts the `users` group to only access rows where the object has been tagged as not containing any personally identifiable information: ```googlesql CREATE ROW ACCESS POLICY no_pii ON my_dataset.my_object_table GRANT TO ("group:users@example.com") FILTER USING (ARRAY_LENGTH(metadata)=1 AND metadata[OFFSET(0)].name="no_pii") ``` ## Security model The following organizational roles are typically involved in managing and using object tables: - **Data lake administrators.** These administrators typically manage Identity and Access Management (IAM) policies on Cloud Storage buckets and objects. - **Data warehouse administrators.** These administrators typically create, delete, and update tables. - **Data analysts.** Analysts typically read data and run queries. Data lake administrators are responsible for creating connections and sharing them with data warehouse administrators. In turn, data warehouse administrators create tables, set appropriate access controls, and share the tables with data analysts. > [!CAUTION] > **Caution:** Data analysts should **not** have the following: > > - The ability to read objects directly from Cloud Storage (see the [Storage > Object Viewer IAM role](https://docs.cloud.google.com/storage/docs/access-control/iam-roles)), which lets data analysts circumvent access controls placed by data warehouse administrators. > - The ability to bind tables to connections (like the BigQuery Connection > Administrator). > > Otherwise, data analysts can create new tables that do > not have any access controls, thus circumventing controls placed by data > warehouse administrators. > - The > [`bigquery.objectRefs.read`](https://docs.cloud.google.com/bigquery/docs/access-control#bigquery.objectRefs.read) > permission on the table's connection (granted by roles such as > [BigQuery > ObjectRef Reader](https://docs.cloud.google.com/bigquery/docs/access-control#bigquery.objectRefReader) and > [BigQuery > ObjectRef Admin](https://docs.cloud.google.com/bigquery/docs/access-control#bigquery.objectRefAdmin)). > > This permission lets data analysts use > [ObjectRef > functions](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/objectref_functions) to read any Cloud Storage object that the connection's > service account can access, circumventing table-level and row-level access > controls. ## Supported object files You can create an object table over any type and size of unstructured data file, and you can create remote functions to work with any type of unstructured data. However, to perform inference by using BigQuery ML, an object table can only be over image files that meet several size and type requirements. For more information, see [Limitations](https://docs.cloud.google.com/bigquery/docs/object-table-inference#limitations). ## Metadata caching for performance You can use cached metadata to improve the performance of inference and other types of analysis on object tables. Metadata caching is especially helpful in cases where the object table is referencing large numbers of objects. BigQuery uses CMETA as a distributed metadata system to handle large tables efficiently. CMETA provides fine-grained metadata at the column and block level, accessible through system tables. This system helps improve query performance by optimizing data access and processing. To further accelerate query performance on large tables, BigQuery maintains a metadata cache. CMETA refresh jobs keep this cache up-to-date. The metadata includes filenames, partitioning information, and physical metadata from files such as row counts. You can choose whether or not to enable metadata caching on a table. Queries with a large number of files and with Apache Hive partition filters benefit the most from metadata caching. If you don't enable metadata caching, queries on the table must read the external data source to get object metadata. Reading this data increases the query latency; listing millions of files from the external data source can take several minutes. If you enable metadata caching, queries can avoid listing files from the external data source and can partition and prune files more quickly. Metadata caching also integrates with Cloud Storage object versioning. When the cache is populated or refreshed, it captures metadata based on the live version of the Cloud Storage objects at that time. As a result, metadata caching-enabled queries read data corresponding to the specific cached object version, even if newer versions become live in Cloud Storage. Accessing data from any subsequently updated object versions in Cloud Storage necessitates a metadata cache refresh. There are two properties that control this feature: - **Maximum staleness** specifies when queries use cached metadata. - **Metadata cache mode** specifies how the metadata is collected. When you have metadata caching enabled, you specify the maximum interval of metadata staleness that is acceptable for operations against the table. For example, if you specify an interval of 1 hour, then operations against the table use cached metadata if it has been refreshed within the past hour. If the cached metadata is older than that, the operation falls back to retrieving metadata from Cloud Storage instead. You can specify a staleness interval between 30 minutes and 7 days. When you enable metadata caching for BigLake or object tables, BigQuery triggers metadata generation refresh jobs. You can choose to refresh the cache either automatically or manually: - For automatic refreshes, the cache is refreshed at a system defined interval, usually somewhere between 30 and 60 minutes. Refreshing the cache automatically is a good approach if the files in Cloud Storage are added, deleted, or modified at random intervals. If you need to control the timing of the refresh, for example to trigger the refresh at the end of an extract-transform-load job, use manual refresh. - For manual refreshes, you run the [`BQ.REFRESH_EXTERNAL_METADATA_CACHE` system procedure](https://docs.cloud.google.com/bigquery/docs/reference/system-procedures#bqrefresh_external_metadata_cache) to refresh the metadata cache on a schedule that meets your requirements. Refreshing the cache manually is a good approach if the files in Cloud Storage are added, deleted, or modified at known intervals, for example as the output of a pipeline. If you issue multiple concurrent manual refreshes, only one will succeed. The metadata cache expires after 7 days if it isn't refreshed. Both manual and automatic cache refreshes are executed with [`INTERACTIVE`](https://docs.cloud.google.com/bigquery/docs/running-queries) query priority. ### Use `BACKGROUND` reservations If you choose to use automatic refreshes, we recommend that you create a [reservation](https://docs.cloud.google.com/bigquery/docs/reservations-intro), and then create an [assignment with a `BACKGROUND` job type](https://docs.cloud.google.com/bigquery/docs/reservations-workload-management#assignments) for the project that runs the metadata cache refresh jobs. With `BACKGROUND` reservations, refresh jobs use a dedicated resource pool which prevents the refresh jobs from competing with user queries, and prevents the jobs from potentially failing if there aren't sufficient resources available for them. While using a shared slot pool incurs no extra cost, using `BACKGROUND` reservations instead provides more consistent performance by allocating a dedicated resource pool, and improves the reliability of refresh jobs and overall query efficiency in BigQuery. You should consider how the staleness interval and metadata caching mode values will interact before you set them. Consider the following examples: - If you are manually refreshing the metadata cache for a table, and you set the staleness interval to 2 days, you must run the `BQ.REFRESH_EXTERNAL_METADATA_CACHE` system procedure every 2 days or less if you want operations against the table to use cached metadata. - If you are automatically refreshing the metadata cache for a table, and you set the staleness interval to 30 minutes, it is possible that some of your operations against the table might read from Cloud Storage if the metadata cache refresh takes on the longer side of the usual 30 to 60 minute window. To find information about metadata refresh jobs, query the [`INFORMATION_SCHEMA.JOBS` view](https://docs.cloud.google.com/bigquery/docs/information-schema-jobs), as shown in the following example: ```googlesql SELECT * FROM `region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT` WHERE job_id LIKE '%metadata_cache_refresh%' AND creation_time > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 6 HOUR) ORDER BY start_time DESC LIMIT 10; ``` To learn more, see [Metadata caching](https://docs.cloud.google.com/bigquery/docs/metadata-caching). For more information on setting metadata caching options, see [Create object tables](https://docs.cloud.google.com/bigquery/docs/object-tables#create-object-table). ## Limitations - Object tables are read-only, because they map to unstructured data objects in Cloud Storage. You can't alter an object table or modify object table data. - Object table support isn't available in Legacy SQL, or other cloud environments such as Amazon Web Services (AWS) and Microsoft Azure. - If you want to perform inference by using BigQuery ML, the model and the object table you use must meet the requirements described in [Limitations](https://docs.cloud.google.com/bigquery/docs/object-table-inference#limitations). - Queries that include object tables can't access more than 10 GB of object metadata. For example, if a query accesses 100 TB from a combination of metadata columns in object tables and object data through signed URLs, only 10 GB of that 100 TB can be from the metadata columns. - Object tables are subject to the same limitations as all other BigQuery external tables. For more information, see [Quotas](https://docs.cloud.google.com/bigquery/quotas#external_tables). - Queries over object tables are subject to the same limitations as all other BigQuery queries. For more information, see [Quotas](https://docs.cloud.google.com/bigquery/quotas#query_jobs). - Remote functions that process unstructured data from object tables are subject to the same [limitations](https://docs.cloud.google.com/bigquery/docs/remote-functions#limitations) as all other remote functions. - Signed URLs generated for the objects in an object table expire after 6 hours, which is the [query execution time limit](https://docs.cloud.google.com/bigquery/quotas#query_jobs). - Inference with BigQuery ML is not supported with on-demand pricing or with the Standard edition. - The following functions are not supported with on-demand pricing or with the Standard edition: - [`ML.CONVERT_COLOR_SPACE`](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-convert-color-space) - [`ML.CONVERT_IMAGE_TYPE`](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-convert-image-type) - [`ML.RESIZE_IMAGE`](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-resize-image) - Object tables can have a maximum of 300 million rows. For optimal listing and query performance, specify a maximum of 10 source URIs in the `uris` option of an object table. Specifying more than 10 bucket URIs reduces the maximum number of files that the object table can support and can cause queries to fail with a `Resources exceeded during query execution` memory error when the total number of files across the buckets is large. If you need to reference more than 10 buckets, split them across multiple object tables. - `UNION ALL` operations that combine both empty and non-empty object tables are not supported and might return an error. - Inside a [VPC Service Controls](https://docs.cloud.google.com/bigquery/docs/vpc-sc) perimeter, AI functions can't process the `ref` column of an object table. The `ref` column always uses the object table's connection as its authorizer, which means that BigQuery generates a signed HTTPS URL for the object, and Gemini Enterprise Agent Platform blocks HTTP and HTTPS fetches for projects inside a perimeter. The function writes the error `INVALID_ARGUMENT: HTTP links are not supported for requests restricted by VPCSC.` to the `status` field of its output. To analyze the object, pass a single-argument [`OBJ.MAKE_REF(uri)`](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/objectref_functions#objmake_ref) value instead, which sends the Cloud Storage URI to the model without generating a signed URL. `OBJ.MAKE_REF` retrieves object metadata from Cloud Storage each time it runs, which might be less scalable for large workloads. For more information, see [Maintaining `ObjectRef` values](https://docs.cloud.google.com/bigquery/docs/objectref-columns#maintaining_objectref_values). ## Costs Costs are associated with the following aspects of object tables: - Querying the tables. - [Refreshing the metadata cache](https://docs.cloud.google.com/bigquery/docs/object-table-introduction#metadata_caching_for_performance). If you have [slot reservations](https://docs.cloud.google.com/bigquery/docs/reservations-workload-management), you are not charged for querying external tables. Instead, slots are consumed for these queries. The following table shows how your pricing model affects how these costs are applied: | | **On-demand pricing** | **Standard, Enterprise, and Enterprise Plus editions** | |---|---|---| | Queries | You are [billed for the bytes processed](https://cloud.google.com/bigquery/pricing#on_demand_pricing) by user queries. | [Slots](https://cloud.google.com/bigquery/pricing#capacity_compute_analysis_pricing) in [reservation assignments with a `QUERY` job type](https://docs.cloud.google.com/bigquery/docs/reservations-workload-management#assignments) are consumed during query time. | | Manually refreshing the metadata cache. | You are [billed for the bytes processed](https://cloud.google.com/bigquery/pricing#on_demand_pricing) to refresh the cache. | [Slots](https://cloud.google.com/bigquery/pricing#capacity_compute_analysis_pricing) in [reservation assignments with a `QUERY` job type](https://docs.cloud.google.com/bigquery/docs/reservations-workload-management#assignments) are consumed during cache refresh. | | Automatically refreshing the metadata cache. | You are [billed for the bytes processed](https://cloud.google.com/bigquery/pricing#on_demand_pricing) to refresh the cache. | [Slots](https://cloud.google.com/bigquery/pricing#capacity_compute_analysis_pricing) in [reservation assignments with a `BACKGROUND` job type](https://docs.cloud.google.com/bigquery/docs/reservations-workload-management#assignments) are consumed during cache refresh.
If there are no `BACKGROUND` reservations available for refreshing the metadata cache, BigQuery automatically uses slots in `QUERY` reservations instead if you are using the Enterprise or Enterprise Plus edition. | You are also charged for storage and data access by [Cloud Storage](https://cloud.google.com/storage/pricing), [Amazon S3](https://aws.amazon.com/s3/pricing/), and [Azure Blob Storage](https://azure.microsoft.com/pricing/details/storage/blobs/), subject to each product's pricing guidelines. When BigQuery interacts with Cloud Storage, you might incur the following Cloud Storage costs: - Data storage costs for the amount of data stored. - Data retrieval costs for accessing data in [Nearline](https://docs.cloud.google.com/storage/docs/storage-classes#nearline), [Coldline](https://docs.cloud.google.com/storage/docs/storage-classes#coldline), and [Archive](https://docs.cloud.google.com/storage/docs/storage-classes#archive) storage classes. Take caution when querying tables or refreshing the metadata cache against these storage classes, as charges can be significant. - Network usage costs for data that you read across different regions, such as when your BigQuery dataset and Cloud Storage bucket are in different regions. - Data processing charges. However, you aren't charged for API calls that are made by BigQuery on your behalf, such as listing or getting resources. ## Using object tables with BigQuery sharing Object tables are compatible with BigQuery sharing (formerly Analytics Hub). Datasets containing object tables can be published as [Sharing listings](https://docs.cloud.google.com/bigquery/docs/analytics-hub-introduction#listings). Sharing subscribers can subscribe to these listings, which provision a read-only dataset, called a [*linked dataset*](https://docs.cloud.google.com/bigquery/docs/analytics-hub-introduction#linked_datasets), in their project. Subscribers can query all tables in the linked dataset, including all object tables. For more information, see [Subscribe to a listing](https://docs.cloud.google.com/bigquery/docs/analytics-hub-view-subscribe-listings#subscribe-listings). ## What's next - Learn how to [create an object table](https://docs.cloud.google.com/bigquery/docs/object-tables). - Learn how to [use object tables to maintain `ObjectRef` columns in standard tables](https://docs.cloud.google.com/bigquery/docs/objectref-columns). - Learn how to [run inference on image object tables](https://docs.cloud.google.com/bigquery/docs/object-table-inference). - Learn how to [analyze object tables by using remote functions](https://docs.cloud.google.com/bigquery/docs/object-table-remote-function).