CSS FeedViz

Updates

• [August 2024]: Introduced multi-threading to process the streaming of CSS Products from the Comparison Shopping Service API to BigQuery.

• [June 2024]: Modified the codebase to stream CSS Products into BigQuery instead of using the insertAll method due to hitting payload size issues.

Overview

CSS FeedViz provides the means for viewing detailed attribute data on disapproved products that have been submitted to the CSS API. The provided information will improve the speed and accuracy with which CSSs will be able to respond to product disapprovals.

Before you begin

As the CSS FeedViz dashboard uses the Comparison Shopping Service API and BigQuery for data retrieval and storage, you need a Google Cloud project and a service account. You will need to enable both the CSS API and BigQuery API in your Google Cloud project.

Required permissions

The service account that you chose to use for CSS FeedViz requires certain access permissions both in Google Cloud and on CSS Center.

Google CSS Center

The email address of the service account needs to be added with Admin access permissions to your CSS group or domain account in Google CSS Center.

Google Cloud

The service account requires rights for getting, creating and updating both datasets and tables within BigQuery. The bigquery.dataEditor predefined IAM role provides adequate permissions for these purposes. In Google Cloud, add the bigquery.dataEditor IAM role to the service account. Instructions on how to manage IAM roles for service accounts can be found at:
https://cloud.google.com/iam/docs/manage-access-service-accounts.

CSS Products Transfer

Configuration Files

CSS Feedviz uses the Comparison Shopping Services API to retrieve products at CSS domain level. Using this API requires a JSON private key file for the service account, which can be obtained from Google Cloud by following the instructions at:
https://cloud.google.com/iam/docs/keys-create-delete#creating

Once you’ve obtained the JSON private key file, rename it as service-account.json and store it in

$(HOME)/css-feedviz/config

In order to provide CSS FeedViz with the relevant CSS Center account information, copy the test account-info.json file from $(HOME)/css-feedviz/config/test to $(HOME)/css-feedviz/config and modify its contents as needed. The file contains a JSON object with the following fields:

Field name	Type	Description
groupId	number	The CSS Group ID.
domainId	number	The CSS Domain ID to retrieve products for.

Transfer Execution

The CSS FeedViz codebase is managed by Maven, with the pom.xml configuration file located in the css-feedviz root directory. To compile the codebase, execute the following command from $(HOME)/css-feedviz.

mvn compile

Once the code has been successfully compiled, execute the following command to transfer product data from the Comparison Shopping Services API to BigQuery.

mvn exec:java -Dexec.mainClass="com.google.cssfeedviz.TransferCssProducts"

Looker Studio Dashboard

To access the CSS FeedViz Looker Studio dashboard template, first join the CSS FeedViz Google Group. Once you’ve joined, make a copy of this Looker Studio template.

Configure the Looker Studio BigQuery Connector to connect to the css_feedviz BigQuery table that has been created by the CSS Products data transfer. This can be done under the Resource > Manage added data sources menu item in Looker Studio. Further information on configuring the Looker Studio BigQuery connector can be found here.

Once properly configured, you should have a dashboard with two populated pages, one with a CSS level attributes and the other with Product level attributes.

Technical Details

TransferCssProducts System Properties

The TransferCssProducts service supports the following Java system properties to allow for greater customisation of functionality.

System property	Default value	Description
feedviz.config.dir	"./config"	Path to directory containing configuration files
feedviz.account.info.file	"account-info.json"	Name of file containing CSS Center account details.
feedviz.dataset.name	"css_feedviz"	Name of the dataset that will be used/created in BigQuery.
feedviz.dataset.location	"EU"	Location of Google Cloud servers where data will be stored/processed.

CSS Products Schema

The schema of the css_products table in BigQuery aligns with the CssProduct resource in CSS API as follows:

Field name				Type	Description
transfer_date				TIMESTAMP	Timestamp of when transfer was initiated.
name				STRING	The name of the CSS Product. Format: "accounts/{account}/cssProducts/{css_product}"
raw_provided_id				STRING	A unique raw identifier for the product.
content_language				STRING	The two-letter ISO 639-1 language code for the product.
feed_label				STRING	The feed label for the product.
attributes				STRUCT	A list of product attributes.
	low_price			STRUCT	Low Price of the aggregate offer.
		amount_micros		INT64	The price represented as a number in micros.*
		currency_code		STRING	The currency of the price using three-letter acronyms according to ISO 4217.
	high_price			STRUCT	High Price of the aggregate offer.
		amount_micros		INT64	The price represented as a number in micros.*
		currency_code		STRING	The currency of the price using three-letter acronyms according to ISO 4217.
	headline_offer_price			STRUCT	Headline Price of the aggregate offer.
		amount_micros		INT64	The price represented as a number in micros.*
		currency_code		STRING	The currency of the price using three-letter acronyms according to ISO 4217.
	headline_offer_shipping_price			STRUCT	The shipping price of the headline offer for the product
		amount_micros		INT64	The price represented as a number in micros.*
		currency_code		STRING	The currency of the price using three-letter acronyms according to ISO 4217.
	additional_image_links			STRING, REPEATED	Additional URL of images of the item.
	product_types			STRING, REPEATED	Categories of the item (formatted as in products data specification).
	size_types			STRING, REPEATED	The cut of the item. It can be used to represent combined size types for apparel items. Maximum two of size types can be provided (see size types).
	product_details			STRUCT, REPEATED	Technical specification or additional product details.
		section_name		STRING	The section header used to group a set of product details.
		attribute_name		STRING	The name of the product detail.
		attribute_value		STRING	The value of the product detail.
	product_weight			STRUCT	The weight of the product in the units provided.†
		value		FLOAT64	The weight represented as a number.‡
		unit		STRING	The weight unit. Acceptable values are: "g", "kg", "oz", "lb".
	product_length			STRUCT	The length of the product in the units provided.§
		value		FLOAT64	The dimension value represented as a number.‡
		unit		STRING	The dimension units. Acceptable values are: "in", "cm".
	product_width			STRUCT	The width of the product in the units provided.§
		value		FLOAT64	The dimension value represented as a number.‡
		unit		STRING	The dimension units. Acceptable values are: "in", "cm"
	product_height			STRUCT	The height of the product in the units provided.§
		value		FLOAT64	The dimension value represented as a number.‡
		unit		STRING	The dimension units. Acceptable values are: "in", "cm"
	product_highlights			STRING, REPEATED	Bullet points describing the most relevant highlights of a product.
	certifications			STRUCT, REPEATED	A list of certificates claimed by the CSS for the given product.
		name		STRING	Name of the certification.
		authority		STRING	Name of the certification body.
		code		STRING	A unique code to identify the certification.
	expiration_date			TIMESTAMP	Date on which the item should expire, as specified upon insertion, in ISO 8601 format.¶
	included_destinations			STRING, REPEATED	The list of destinations to include for this target (corresponds to checked check boxes in Merchant Center). Default destinations are always included unless provided in excluded_destinations.
	excluded_destinations			STRING, REPEATED	The list of destinations to exclude for this target (corresponds to unchecked check boxes in Merchant Center).
	cpp_link			STRING	URL directly linking to the Product Detail Page of the CSS.
	cpp_mobile_link			STRING	URL for the mobile-optimized version of the Product Detail Page of the CSS.
	cpp_ads_redirect			STRING	Allows advertisers to override the item URL when the product is shown within the context of Product Ads.
	number_of_offers			INT64	The number of aggregate offers.
	headline_offer_condition			STRING	Condition of the headline offer.
	headline_offer_link			STRING	Link to the headline offer.
	headline_offer_mobile_link			STRING	Mobile Link to the headline offer.
	title			STRING	Title of the item.
	image_link			STRING	URL of an image of the item.
	description			STRING	Description of the item.
	brand			STRING	Brand of the item.
	mpn			STRING	Manufacturer Part Number (MPN) of the item.
	gtin			STRING	Global Trade Item Number (GTIN) of the item.
	google_product_category			STRING	Google's category of the item (see Google product taxonomy).#
	adult			BOOL	Set to true if the item is targeted towards adults.
	multipack			INT64	The number of identical products in a merchant-defined multipack.
	is_bundle			BOOL	Whether the item is a merchant-defined bundle. A bundle is a custom grouping of different products sold by a merchant for a single price.
	age_group			STRING	Target age group of the item.
	color			STRING	Color of the item.
	gender			STRING	Target gender of the item.
	material			STRING	The material of which the item is made.
	pattern			STRING	The item's pattern (e.g. polka dots).
	size			STRING	Size of the item.
	size_system			STRING	System in which the size is specified. Recommended for apparel items.
	item_group_id			STRING	Shared identifier for all variants of the same product.
pause			STRING	Publication of this item will be temporarily paused.
custom_label_0			STRING	Custom label 0 for custom grouping of items in a Shopping campaign.
custom_label_1			STRING	Custom label 1 for custom grouping of items in a Shopping campaign.
custom_label_2			STRING	Custom label 2 for custom grouping of items in a Shopping campaign.
custom_label_3			STRING	Custom label 3 for custom grouping of items in a Shopping campaign.
custom_label_4			STRING	Custom label 4 for custom grouping of items in a Shopping campaign.
css_product_status				STRUCT	The status of a product, data validation issues, that is, information about a product computed asynchronously.
	destination_statuses			STRUCT, REPEATED	The intended destinations for the product.
		destination		STRING	The name of the destination
		approved_countries		STRING, REPEATED	List of country codes (ISO 3166-1 alpha-2) where the aggregate offer is approved.
		pending_countries		STRING, REPEATED	List of country codes (ISO 3166-1 alpha-2) where the aggregate offer is pending approval.
		disapproved_countries		STRING, REPEATED	List of country codes (ISO 3166-1 alpha-2) where the aggregate offer is disapproved.
	item_level_issues			STRUCT, REPEATED	A list of all issues associated with the product.
		code		STRING	The error code of the issue.
		servability		STRING	How this issue affects the serving of the aggregate offer.
		resolution		STRING	Whether the issue can be resolved by the merchant.
		attribute		STRING	The attribute's name, if the issue is caused by a single attribute.
		destination		STRING	The destination the issue applies to.
		description		STRING	A short issue description in English.
		detail		STRING	A detailed issue description in English.
		documentation		STRING	The URL of a web page to help with resolving this issue.
		applicable_countries		STRING, REPEATED	List of country codes (ISO 3166-1 alpha-2) where the issue applies to the aggregate offer.
	creation_date			TIMESTAMP	Date on which the item has been created, in ISO 8601 format.
	last_update_date			TIMESTAMP	Date on which the item has been last updated, in ISO 8601 format.
	google_expiration_date			TIMESTAMP	Date on which the item expires in Google Shopping, in ISO 8601 format.

* 1 million micros is an equivalent to one's currency standard unit, for example, 1 USD = 1000000 micros.

† The value must be between 0 (exclusive) and 2000 (inclusive).

‡ The value can have a maximum precision of four decimal places.

§ The value must be between 0 (exclusive) and 3000 (inclusive).

¶ The actual expiration date is exposed in css_product_status as google_expiration_date and might be earlier if expiration_date is too far in the future. Note: It may take 2+ days from the expiration date for the item to actually get deleted.

# When querying products, this field will contain the user provided value. There is currently no way to get back the auto assigned google product categories through the API.