Miva 10.01.00 greatly expands on Miva’s MySQL Full Text Searching features. These new updates give you more granular control over how each individual field is searched and whether Full-Text is used or not on a per field basis. This level of control solves previous issues where using Full-Text Search across all searchable fields creates undesired results. For example, MySQL Full Text has a 4-character limit, so any searches for 3 characters or less did not return any results. These new controls give you the merchant the control to determine the best type of search for each field depending on your data set
In addition, there is a brand-new sorting option for “Relevance”. If “Sort By Relevance” is enabled for any search field, the search results returned are then sorted by a relevance score calculated by the database and an optional weight can be given to boost the score. For example, if you wanted to give weight to search term matches in the product name over matches from the description this field weight allows you to do exactly this.
Optimizing Search Results
There is now a "Searchable Fields" dashboard to control which fields are search as well as how they are searched.
Here you can enable and disable the searchable fields and see what module is making that field available to search.
There is a new setting to set the Max searchable terms. This defaults to 10. For example is someone searches for “Blue Pants” that would get split into 2 terms. Increasing this beyond the default of 10 is allowed but setting this limit too high could have negative search performance because the number of queries which need to be run.
This setting is new and gives you granular control over how each field is searched. The default search is Contains (term) and is the same method Miva has used to search all fields in past versions of the software. All fields will have this search type initially. This mean that the search string is split into individual words (terms) and all the terms must appear in the fields being searched. For example, if the search string was “White Wedding Dress” all 3 terms must appear in the search fields, but not as an exact match. Each term can be a substring of another word and the order in which they appear does not matter.
There are 4 different Search Types
- 1. Exact Match – The entire search string must exist in the search fields, exactly as searched, ex: “Blue Jersey”
- 2. Contains - The entire search string as entered is searched for and can be a substring of another term.
- 3. Contains (Term) - The search string is split into individual words (terms) and all the terms must appear in the fields being searched, in any order including as a substring.
- 4. Full Text – A natural language comparison search is performed. https://dev.mysql.com/doc/refman/8.0/en/fulltext-natural-language.html
Note: There is a upcoming performance improvements coming in 10.01.02 to optimize queries for searches which include spaces when "contains (terms)" is being used as the search type and custom fields are being searched. A short term solution is to use type of "Contains" vs "Contains (Term)" which treats the entire search as a single string.
When to Use Full Text Search
To answer this question, we need to understand a bit more about Full Text search and its benefits over a regular search query.
First in order to use MySQL Full Text search, you need to be on MySQL 5.6 or higher. If you’re not seeing the Full Text options in your Miva admin, contact our support team.
Full-text search is a technique that enables you to search for records that might not perfectly match the search criteria. It works by creating indexes of the search data on the table. When these index are generated certain words are excluded. These are called stop words.
A natural language search interprets the search string as a phrase in natural human language (a phrase in free text). There are no special operators, with the exception of double quote (") characters. The stopword list is applied to the search query.
“More specifically, FTS retrieves documents that don’t perfectly match the search criteria. Documents are database entities containing textual data. This means that when a user searches for “cats and dogs”, for example, an application backed by FTS is able to return results which contain the words separately (just “cats” or “dogs”), contain the words in a different order (“dogs and cats”), or contain variants of the words (“cat” or “dog”). This gives applications an advantage in guessing what the user means and returning more relevant results faster”. [Reference]
Any word that is too short is ignored. The default minimum length of words that are found by full-text searches is three characters for InnoDB search indexes, or four characters for MyISAM. This means that if you have a common words less than 4 characters that is important to be searched you should not enable Full Text Search on that specific field.
Miva Full Text implements MySQLs Full Text Natural Language Comparison
Full Text Natural Language Mode
A natural language search type - such a search mode interprets the search string as a literal phrase
In general, we recommend Full Text Search be used on fields such as product description and product name, where you don’t need an exact string match. It should not be used on fields such as SKU or product attributes where you typically do want an exact match.
Sorting Search Results
Miva 10.01 introduces a brand new sorting called “relevance.” This sort is applied to the result-set to determine the order in which the products are returned.
How is Relevance Defined?
Relevance is computed based on the number of words in the row (document), the number of unique words in the row, the total number of words in the collection, and the number of rows that contain a particular word.
Every correct word in the collection and in the query is weighted according to its significance in the collection or query. Thus, a word that is present in many documents has a lower weight, because it has lower semantic value in this particular collection. Conversely, if the word is rare, it receives a higher weight. The weights of the words are combined to compute the relevance of the row.
Note: The term “document” may be used interchangeably with the term “row”, and both terms refer to the indexed part of the row. The term “collection” refers to the indexed columns and encompasses all rows.
This value is a multiplier that gets applied to the relevance score calculated in the sort. You could simplify relevance down to how often a given word appears in a search field compared to the dataset as a whole. So, adding a multiplier to this score is saying that “I want to boost results where the searched terms appear more frequently”
This is a decimal value with the default being no boosting, or 1.00. If you wanted changed this value to 1.5, for the name field, it means that it would give this field a 50% boost when calculating the relevance score allowing those results which have a high frequency in the name field to be sorted first.
Customer Defined Sort By Relevance
If you want offer customers the ability to sort by Relevance on Search Pages manually as an option in the drop-down menu you will need to add “relevance” as a sorting option:
Miva Search vs Power Search (module)
If you are currently using Power Search (Emporium Plus Module) to control your search results, the new features in 10.01 will allow you to replace existing functionality found in the module including:
- - Relevance Sorting
- - Field Boosting
- - Full Text Searching on specific fields
The added benefits of migrating off the Power Search module include:
- - No separate search index to rebuild
- - No limit on custom field name length or special characters
- - No Search Log to Clear