Product Data Fields and Feeds

Released In: 10.04.01

Overview

The Product Data Fields and Feeds module, released as part of Miva 10.04.01, provides a mechanism for template-based generation of complex product data upon modification of the data. This is fundamentally different than our standard Template Based feeds module which loads all products and iterates though them one at a time to generate the output and write it to a file/feed.

By queueing each product as it changes and doing the output generation on change, it allows the final generation of the feed to happen in seconds vs. minutes or hours since all the heavy lifting has already been done ahead of time.

The regeneration of the feed can be triggered by any number of product modifications, and is accomplished through a scheduled task and a queue that tracks modified products. This is more efficient and provides for faster feed file generation.

It also has the ability to hand off the updated data to Miva’s new Searchspring Live Indexing and Pricing integration. For additional information on Searchspring, see here.

The primary goal of this module is to allow for extremely fast and efficient output of large dataset feeds which can then be consumed by other external systems. A secondary use case is to precalculate dynamic data on change of each product record so they it can be loaded as static data for faster performance in runtime.

On this Page

Installation

Product Data Fields and Feeds is a new module, so you will have to install it the initial time you use it.

Click the Settings header on the left side of the page, then the Modules subhead.

On the modules page, scroll or just search for the Product Data Fields and Feeds module, then click Install.

Alt text here

Use/Configuration

To use the module, click the Settings header on the left side of the page, then click the Utilities subhead.

On the page that opens, click the Product Data Fields tab, and then click the Add Field button.

In the box that opens you will give the new field a name, and a code.

Then, you will set the actions for the field based on what changes have been made to the product.

The changes are:

Alt text here

Edit Field: Set the desired action for each of the changes.

Each change has the same three options for what action is taken. The actions are:

Trigger: The trigger field is an optional field that gives you more flexibility for when the queue is processed. When one of the actions above occur, the module will add the product(s) to the Product Data Fields and Feeds Queue. This queue gets processed by a scheduled task. This scheduled task can be setup to run at an interval, for example once an hour or it can be setup to run based on a trigger.

If you want it to be based on a trigger, instead of waiting for the scheduled task to run (based on its schedule) you can have the Product Data Fields and Feeds module initiate the scheduled task to process the queue as changes to products are happening. This allows the queue to be processed closer to real time.

Public: This flag is to determine if thes values are acessable as public custom fields. If public they will be accessable via the JSON API.

Using a Trigger to Process the Queue

To set this up, first, create the scheduled task. While configuring the scheduled task, give the task a trigger name.

For the Scheduled Task Operation choose "Product Data Fields and Feeds: Generate Product Field Data." Alt text here

The trigger name used in the Scheduled Task can then be added to the trigger field in the Product Data Fields and Feeds module.

Alt text here

This configuration will cause the queue to be processed as product changes happen (with an optional delay, if configured).

Page Template Created

Every new product data field creates a new page template which it uses to generate the data. The default code for the page is PRODUCTDATAFIELD_CODE. The "CODE" section will be equal to the name of the product data field.

To get to the page template, after the Data Field has been created, from the batch list select the record then you'll see an edit template button which will take you directly to the page template to edit.

Edit Product Data Field Page Template

Select the record from the batch list by checking the checkbox on the left side of the row

Page Template Configuration Options

Example of a default template. It is empty by default with no output.

Alt text here

There are individual options to turn on additional data to be available to the template. For performance reasons, you should only enable the options to load the data you need.

All the data in the template can be referenced in the l.settings:product:xxx with naming consistent with other component modules.

Scheduled Task

The field data generation is accomplished through a scheduled task. To create the scheduled task, click Store Settings on the left side of the page, then the Scheduled Tasks tab. The generation of the product field data can be scheduled to run at various time, or when various things happen, like a change in inventory of a product, for example.

Product Data Fields - Schedueld Task Configuration

The seheduled task is not created on install and needs to be setup for the queue to be processed properly

Product Data Field: This is the code of the product data field you want to process. This is setup in the Use/Configuration step of the module
Generation: There are three options in this drop down:

Process No More than X Products: This setting allows you limit how many records in the queue you process in any given execution. This can be useful if you have another process you want to run, such as importing data to Searchspring.

Process No More than X Seconds: This setting allows you limit how long the task will run processing the queue.

Trigger On Completion: This setting allows you to call another trigger when this scheduled task is completed. This allows you to hand over execution to another task you want to run directly after this one, such as a Searchspring task to push products to Searchspring.

Queue

When a product is updated, such as the inventory amount of a product changes, that product is then added to queue for regeneration. The queue is processed based on the scheduled task and can be tied to the changes listed above. You can see if a product is in the queue to be processed on the product batch list screen. There are three custom fields made available by the module to determine if the product is in the queue to be processed, when it was last processed and the data that was generated after it was run though the template.

Batch List Custom Fields

These three batch list fields can be added to the product batch list to see if a product is in the queue to be updated, when it was last updated and the resulting data that was generated. This data is what ultimately gets written to the feed when it is generated.

Feed Generation Settings

To set up the feed, click the Marketing header, then click the Feeds tab on the page that opens. For additional information on feeds, and how to create them, visit: Miva Feeds Overview

To write the content generated by the Product Data Field Module to a file you need to also create a feed for it. Navigate to Marketing -> Feeds -> Add Feed

Create Feed File

Make sure you select Product Data Fields and Feeds as the Feed Module, For Product Data Field, this should contain the field you which to export into the feed.

Feed Page Template

Creating a feed will generate a new page template with the prefix PRODUCTDATAFEED_XXXX. This is the page template that is used to generate the final output file. Below is the default template which writes the file:

This template can optionaly be edited, but out of the box will take the individual generated data for each product and write it to the file name and location you specify

Product Data Feed - Template Variables

Using with Searchspring

CONTENT TO COME

JSON API Functions

This module surfaces a JSON API function to allow an external system to mark a product as modified. It allows you to mark a product as modified (add it to the Queue for processing), delete a product from the queue as well as trigger the queue to be processed. The Product Data Fields and Feeds module must be installed in order to be used.

https://docs.miva.com/json-api/functions/productmodified

Note

Note: Because this is a module level function you'll need to use the following format to whitelist the function for your API token:

Module:productdata:ProductModified

Provisioning

Looking for Developer Docs?

We have a whole section for that, including: Developer Training Series, Template Language docs, Module Development tutorials and much, much more.

Head to the Developer Section

This website uses cookies to identify visitors, track visitors to our website, store login session information and to remember your user preferences. By continuing to use this site you agree to our use of cookies. Learn More.

This website uses cookies. By continuing to use this site you agree to our use of cookies. Learn More.

Accept

Copyright © 1997 – 2021 Miva©, Miva Merchant©, MivaPay©, MivaCon© Miva, Inc. All Rights Reserved.