From Movable Type

Contents 1 XSearch: A Proposal for Extensible Search for Movable Type 1.1 Purpose 1.2 Registry Definition 1.2.1 Synopsis 1.2.2 Keys 1.2.3 Additional Search Parameter Definition 1.3 Standard Query Parameters 1.4 MT::XSearch 1.5 MT::XSearch::Results 1.6 Plugin Methods 1.7 Default Link Builder 1.7.1 Default Link Builder Modifiers 1.8 Template Tags 1.9 Other

XSearch: A Proposal for Extensible Search for Movable Type

"One search engine does not fit all."

XSearch is a proposal for the implementation of an extensible search framework in Movable Type that would enabled varying search mechanisms to be implemented as plugins to the system.

Purpose

• Enable any local or external search engine to search anything
• A single common script (access point) for all searching with standard query parameters
• Common shared template tags for presenting and navigating any results
• Define a standard set of query parameters while allowing for additional plugin specific parameters to be implemented.
• Provide common mechanism for caching and handling error

Registry Definition

Synopsis

search:
    'foo':
        search: 'MT::Plugin::Foo::Search::search' 
        stash: 'MT::Plugin::Foo::Search::stash'
        link_builder: 'MT::Plugin::Foo::Search::link_builder'
        result_count: 'MT::Plugin::Foo::Search::result_count'
        parameters:
            req_example_param:
                required: 1
            opt_example_param:
                default: 'foo'
            opt_example_param_no_default: ~

Keys

• search

The 'search' key is the root element for all search plugins. This element is typically the child of a plugin's root element.

• search key

The search key is a unique identifier string used by the system to locate and load a specific search implementation. In the synopsis above 'foo' is the search key.

• search

A code reference or handler that executes the search based on the standard and plugin specific parameters. This key must be defined by all search plugins.

• stash

A code reference or handler that creates the context for one result row. This key must be defined by all search plugins.

• link_builder

A code reference or handler that generates a URL such as those used to navigate through results. This key is optional. If undefined a default link builder will be used. See "Default Link Builder."

• result_count

Code referwnce or handler that returns the result count. this is important to paginated views because you don't want to load the entire set. If this routine is not defined, XSearch will use the count of elements in the result set array returned by search.

Additional Search Parameter Definition

A specific plugin may choose to recognize or even require additional parameters that are applicable to implementing a certain type of search.

QUESTION: More guidance here?

• parameters

An optional registry key that enables the definition of additional parameters in addition to the standard set. The key and underlying branch is optional. By omitting it, the framework will work with the standard query parameter set. This registry key is a child of the search key.

• parameter name

The name of the query parameter that is being defined. A child of a parameters key.

• required

An boolean flag that a required parameter to perform a query with this plugin. The default value if it is not present is 0 thereby making a parameter optional. A child of a parameter name key.

• default

A string value that should be used if the parameter is not defined. A child of a parameter name key.

Define how standard query parameters that are redefined in the registry should be handled. Error? Modify? Warn?

Standard Query Parameters

To enable the potential for drop-in search replacements and a loose standard in search plugin design, the framework implements a standard set of query parameters.

Note that any parameters value is case insensitive though all lowercase strings are preferred. A specific plugin may not choose to follow this convention though it is highly recommended.

• key

REQUIRED. A string that identifier which search plugin should be used to generate results.

• search

REQUIRED. The query string to search for. Treat as "no search" if not present or a null string and NOT an error.

• sort_on

OPTIONAL. A column name, key or other identifier that can be used to determine how to sort the results. All plugins are recommended to implement a default value if sorting is an option at all.

• sort_order

OPTIONAL. Defines the direction of the sort with a value of either "ascend" or "descend".

• offset

OPTIONAL. Integer value that defines the number of results to skip from the start of the sorted array. Similar to the offset argument in mt:entries. Plugins should assume a default value of 0, do not skip any results.

• limit

OPTIONAL. Integer value that defines the maximum number of results to return and display. This parameter is similar to the lastn argument mt:entries implements. A default value should be implemented by the plugin. Allowing all results to be returned is not recommended.

MT::XSearch

An object that initializes and contains a search plugins functional parts.

• MT::XSearch->new($search_key);

Instantiates the objet and calls the init routine passing through the required search key value

• $xsearch->init(@_)

Called by the new method passing through its parameters, this method retrieves the search plugin's registry entry. The registry entry is used to intialize the object with the various handlers and parameter defintions used in processing a search request.

• MT::XSearch->execute(\%search_params||$can_param_object)

The cornerstone operation of the XSearch framework that takes users search queries and returns results accordingly.

• Extract the search-specific params from passed HASH or object then can param.
• Checks if search_key has been defined or throw error.
• Instantiates a new XSearch object. (QUESTION: Cache these?)
• Check for missing params and return an error
• If the search param is not defined or empty return a results object with a count of zero. (Don't bother calling the plugins search method.)
• Call the plugin's search method.
• Return the results object or error message

NOTE: The calling application handles initialize and managing any stashed values and references.

• $xstash->search(\%search_params)

Executes the search method define by the search plugin's search code handler defined in the registry. See the search method listed under Plugin Methods for implementation details. Developers typically will use the execute method rather then call this method method directly.

• $xsearch->required_params

Returns an ARRAY containing all the required parameters for the plugin.

This list is generated by merging the standard XSearch query parameters with any parameters defined in the plugins registry entry.

• $xsearch->optional_params

Returns an ARRAY containing all the optional parameters for the plugin.

This list is generated by merging the standard XSearch query parameters with any parameters defined in the plugins registry entry.

• $xsearch->params

Returns an ARRAY containing all recognized parameters for the plugin. A concatentation of the returned arrays of the required_params and optional_params methods.

MT::XSearch::Results

An object container that holds the results and related metadata from a search performed by a plugin.

• MT::XSearch::Results->new($param,$results,$count)

Typically this method will be used by the execute method in MT::XSearch to create an object to store the results, count and search parameters used to generate the results.

• $results->args

Returns a HASH reference with the parameters and values used to generate the result set.

QUESTION: Should be cloned since this array is supposed to be read only?

QUESTION: Use of "params" instead of args here or will that be too confusing to the params in MT::XSearch?

• $results->search_string

Returns the string used to execute the search -- essentially $results->args->{search}. If the search parameter was not defined (no search was performed) this method returns an empty string.

• $results->key

Return the key of the search plugin -- essentially $results->args->{search_key}.

QUESTION: Or should this return an existing XSearch object that has been instaniated?

• $results->results

Returns an ordered ARRAY reference received from the search. The set is constrained by parameters such as offset and limit and is not necessarily all the potential results.

QUESTION: Should we work MT::Promise into here?

• $results->result_count

Returns an integer value of the total number of matching records that were found while executing the search. This number is not necessarily the number of records returned by the results method.

• $results->link_builder($ctx[,%args])

Creates a URL using the search plugin's link_builder code handle that is defined in the registry. If this registry key was not defined, XSearch will use the default link builder. See the link_builder method listed under Plugin Methods for implementation details. Developers typically will use the execute method rather then call this method method directly.

• $results->missing

Returns an array of missing required parameters from the submitted search. This method will return undef if all required parameters have been submitted. This method will typically to produce more detailed error messages and feedback.

• $results->errstr

The result set object is a subclass of MT::ErrorHandler and inherits its errstr method that returns a string describing the error.

• $results->error

The result set object is a subclass of MT::ErrorHandler and inherits its error method that sets the object or class error string and returns undef.

Plugin Methods

This section details the methods that a plugin will implement to manage and execute the search in the system.

• $xstash->search(\%search_params||$can_param_object)

This method is passed a hash reference containing all required and optional parameters defined by the plugin.

The search method should not assume an HTTP request as the source of a search request coming in. While an HTTP request is the most common use, adhereing this assumption will enable search plugins to be potentially used by tasks, command-line tools or as a resource to some other plugin.

Developers also have the option of passing in an object with a param method that functions like the CGI module.

Returns an MT::XSearch::Results object if the search was performed without error. This includes a search that yields zero results or a "no search" condition.

• $xsearch->stash($results,$ctx)

This method is called with the context object being used to build a page and enable a search plugin to create the necessary context for one result row.

Nothing of value is returned by this method and should be ignored.

• $results->result_count

This method receives the context object being used to build a page and returns the total search results as an integer. Note that the count is NOT the number of records being displayed on the page.

• $results->link_builder($ctx[,%args])

This method receives the context object being used to build a page and an optional HASH containing parameters that will modify the link being generated. See the "Default Link Builder" section for the list and associated function of the arguments.

This method must return a fully qualified HTTP or HTTPS URL with query string for navigating the results of the search.

Default Link Builder

The default link builder is used to generate URLs if a search plugin does not define one. This method should be able to generate all links used by the standard sample tags for navigating search results in a browser.

The link generated by this method is the concatenation of the values of the CGIPath and SearchScript configuration directives along with the current search parameters modified by optional arguments.

If called with an optional HASH, that is used to modify the current search parameters[1] according to their designated function. The arguments recognized by this default handler is detailed below. It is recommended that any search plugin supplying its own link builder support these arguments. These result of modifying the existing search parameters with the arguments is then used to generate an HTTP query string.

[1] IMPORTANT: We have to be working with a copy of the params in here.

Only required and optional parameters will be used to create the query parameters. If an optional parameter has a default value defined and its value is the same as the one being used to built the link it will be omitted.

NOTE: The link should be built using the URI package from CPAN.

QUESTION: Make the default link builder an exportable method since we are trying to avoid getting into subclassing and inheritance? A search plugins own link builder should be able to utilize this method if it wishes.

Default Link Builder Modifiers

• previous => 1

Subtract limit from offset and use as new offset. Treat as first if the new offset is less then zero.

• next => 1

Add limit to offset and use as new offset. Treat as last new offset is great then the result count.

• first => 1

Delete offset query parameter. (Offset equals zero, its default.)

• last => 1

The new offset is the result count mod the limit.

• page => $val

Set offset to limit * $val. Treat as last page if the new offset is greater than the result count.

• all => 1

Delete offset AND limit parameters.

QUESTION: Is this a good idea given the potential performance problems this could genrate?

Template Tags

• mt:ifsearchblogcontext

A conditional tag that displays its contents if a weblog is in context. This tag is useful for creating one template that can be reused for weblog and system tag search.

• mt:ifsearchresults

A conditional tag that displays its contents if any search results were returned.

• mt:searchresults

A container tag for all the search results and navigation.

• mt:searchresultcount

Inserts the total number of results returned from the search.

• mt:searchstring [decode_url=0|1]

Inserts the search string used to fetch the results.

• mt:searchheader

A container tag that renders its contents before the search results.

• mt:searchfooter

A container tag that renders its contents after the search results.

• mt:nosearch

A container tag the renders its contents if the tempalte was loaded without a search being performed.

• mt:nosearchresults

A container tag that renders its contents if no results were returned from the search.

• mt:searchnavigation

Container tag that holds all of the following navigational tags.

• mt:searchpreviouslink

Generates a URL to the previous page of results.

• mt:searchnextlink

Generates a URL to the next page of results.

• mt:searchfirstlink

Generates a URL to the first page of results.

• mt:searchlastlink

Generates a URL to the last page of results.

• mt:searchalllink

Generates a URL to display all results in one page.

• mt:searchpreviousnotfirst

A conditional tag that only renders its contents if the previous page is not the first page of results.

• mt:searchnextnotlast

A conditional tag that only renders its contents if the next page is not the last page of results.

• mt:searchhasprevious

A conditional tag that only renders its contents if the current page has a predecessor.

• mt:searchhasnext

A conditional tag that only renders its contents if there are more results (pages) after the current page.

• mt:searchpagelinks

Creates a context for generating a list of links to a page. Pages are based on the limit and offset params applied to the search.

• mt:searchpageindex

Inserts the page number (an integer).

• mt:searchpagelink

Inserts a URL to generate the page number (managed by mt:searchpagelinks) currently in context.

• mt:searchpageindexnotcurrent

A conditional tag that renders its contents if the mt:searchpagelinks index in context does not match the current page of results being rendered.

• mt:searchpageindexcurrent

A conditional tag that renders its contents if the mt:searchpagelinks index in context matches the current page of results being rendered.

Other

Should the framework implement a maximum limit to avoid massive result sets being loaded up if limit was "gamed" by the caller/user?