Expose search engine to external tools

[BjarniRunar]

This issue tracks the milestones defined in the "Mailpile Search Integration" Project which is supported by a grant from NLNet and NGI0 Discovery.

• Port over / re-implement a working search engine in Python 3
• Enhance the search engine to allow partitioning / compartmentalization of result sets
• Give the user tools to selectively grant access to one or more partitions of results
• Expose the search engine using a documented web-based API
• Describe how to configure SearX for use with moggie, as a proof of concept
• Review and update documentation when Behave as a drop-in replacement for notmuch #3 is done

[BjarniRunar]

The search engine and partitioning work is (modulo bugs and future refactoring) complete.

Some notable milestones along the way:
- bf91f62 - Create numpy-based IntSet for tags and posting lists
- 444eecc - This might be a search engine
- 68db149 - 1st draft of working partial word search
- 7a89cc1 - Add support for All and subtraction in search
- a049a2d - Parse text into a search-op tree
- f546aa7 - WIP: Search worker
- 15fbc90 - Move search engine logic to engine.py
- 355acab - Add namespace compartmentalisation support to the search engine
- bd3d647 - Milestone: Adding to index, and searching for all:mail works!
- bd8ee3a - Importing mail and searching works
- 64439ab - Search: Progress on tags, add short-word-pair binding
- b57575f - Search engine enhancements, tagging progress
- a630905 - Fix some tag namespacing and search scoping buglets

[BjarniRunar]

There are now tools and docs explaining how to grant restricted access to the moggie search engine:

• https://github.com/BjarniRunar/moggie/blob/master/docs/search-integration.md

More relevant commits:

• 47ee0b2 - Improve how contexts scope searches
• 2ee4f2f - Make app load port and kite details from config
• 55f5cdf - Added ConfigSet/ConfigGet API calls, moggie grant now works
• d373426 - Added list edits to ConfigSet, moggie context now works
• 6ce8ed6 - Select default context per access grant...
• 8594f5f - Created docker-based blackbox test script for context/grant...
• 2fde3ef - Create docs/search-integration.md

[BjarniRunar]

There is now enough of a web interface exposed to allow 3rd party tools to search, display results, and display individual messages.

The search integration docs have been updated with a section aimed at developers: https://github.com/BjarniRunar/moggie/blob/master/docs/search-integration.md#6-notes-for-developers

Relevant commits:

• 8ec8131 - moggie search: Add signatures to result message IDs
• 48bbd69 - moggie show mostly works now
• a25782c - Documented search/show API for integration

[BjarniRunar]

The proof-of-concept SearX plugin is here: https://github.com/BjarniRunar/moggie/blob/master/contrib/searx/moggie.py

It works! However there does seem to be a bug in moggie's web server causing it to fail when Firefox connects to it directly (when a user clicks a search result in SearX). Reloading the blank page seems to suffice? I'll have to track that down, but it's not directly related to this.

[BjarniRunar]

As a first draft, this is complete. Leaving open, because the API should be reviewed and documentation improved further once all the notmuch #3 work is complete.