FEATURE: Add documentation of current state

* Document all options
* Document concepts
* Document connections and indexer

Documentation/source/concepts.rst

@@ -0,0 +1,30 @@
+.. _concepts:
+
+Concepts
+========
+
+The extension is built with the following concepts in mind.
+
+.. _concepts_connections:
+
+Connections
+-----------
+
+It should be possible to use different search services like elasticsearch and solr out of the box.
+If a service is not contained, it should be possible to implement the necessary part by implementing
+the necessary interfaces and configuring the extension to use the new connection.
+
+Also it should be possible to use multiple connections at once. This way multiple search services
+can be used in the same installation.
+
+Currently only :ref:`Elasticsearch` is provided.
+
+.. _concepts_indexing:
+
+Indexing
+--------
+
+The indexing is done by one of the available indexer. It should be possible to define the indexer to
+use for certein document types. Also it should be possible to write custom indexer to use.
+
+Currently only the :ref:`TcaIndexer` is provided.

Documentation/source/conf.py

@@ -167,7 +167,7 @@ html_static_path = ['_static']
 
 # Custom sidebar templates, maps document names to template names.
 html_sidebars = {
-    "**": ['about.html', 'globaltoc.html', 'localtoc.html', 'donate.html', 'searchbox.html']
+    "**": ['about.html', 'globaltoc.html', 'localtoc.html', 'donate.html', 'searchbox.html', 'relations.html']
 }
 
 # Additional templates that should be rendered to pages, maps page names to
@@ -301,4 +301,6 @@ texinfo_documents = [
 
 
 # Example configuration for intersphinx: refer to the Python standard library.
-intersphinx_mapping = {'https://docs.python.org/': None}
+intersphinx_mapping = {
+    't3-tca-ref': ('https://docs.typo3.org/typo3cms/TCAReference/', None)
+}

Documentation/source/configuration.rst

@@ -0,0 +1,163 @@
+.. highlight:: typoscript
+
+.. _configuration:
+
+Configuration
+=============
+
+The extension offers the following configuration options through TypoScript. If you overwrite them
+through `setup` make sure to keep them in the `module` area as they will be accessed from backend
+mode of TYPO3. Do so by placing the following line at the end::
+
+    module.tx_searchcore < plugin.tx_searchcore
+
+.. todo::
+
+   We will use references inside the extension to make the above unnecessary in the future.
+
+Currently no constants are available, but this will change in the near future to make configuration
+easier.
+
+The strucutre is following TYPO3 Extbase conventions. All settings are placed inside of::
+
+    plugin.tx_searchcore.settings
+
+Here is the example default configuration that's provided through static setup:
+
+.. literalinclude:: ../../Configuration/TypoScript/setup.txt
+   :language: typoscript
+   :linenos:
+   :caption: Static TypoScript Setup
+
+.. _configuration_options:
+
+Options
+-------
+
+The following section contain the different options for e.g.
+``plugin.tx_searchcore.settings.connection`` or ``plugin.tx_searchcore.settings.index``.
+
+.. _configuration_options_connection:
+
+connection
+^^^^^^^^^^
+
+Holds settings regarding the connection to search service like elasticsearch or solr.
+
+Configured as::
+
+    plugin {
+        tx_searchcore {
+            settings {
+                connection {
+                    // the settings
+                }
+            }
+        }
+    }
+
+.. _host:
+
+``host``
+""""""""
+
+    Used by: :ref:`Elasticsearch`.
+
+    The host, e.g. ``localhost`` or an ip where the search service is reachable from TYPO3
+    installation.
+
+    Example::
+
+        plugin.tx_searchcore.settings.connection.host = localhost
+
+.. _port:
+
+``port``
+""""""""
+
+    Used by: :ref:`Elasticsearch`.
+
+    The port where search service is reachable. E.g. default ``9200`` for elasticsearch.
+
+    Example::
+
+        plugin.tx_searchcore.settings.connection.port = 9200
+
+
+
+.. _configuration_options_index:
+
+index
+^^^^^
+
+Holds settings regarding the indexing of TYPO3 records to search service.
+
+Configured as::
+
+    plugin {
+        tx_searchcore {
+            settings {
+                index {
+                    // the settings
+                }
+            }
+        }
+    }
+
+.. _allowedTables:
+
+``allowedTables``
+"""""""""""""""""
+
+    Used by: :ref:`TcaIndexer`.
+
+    Defines which TYPO3 tables are allowed to be indexed. Only whitelisted tables will be processed
+    through Command Line Interface and Hooks.
+
+    Contains a comma separated list of table names. Spaces are trimmed.
+
+    Example::
+
+        plugin.tx_searchcore.settings.index.allowedTables = tt_content, fe_users
+
+.. _rootLineBlacklist:
+
+``rootLineBlacklist``
+"""""""""""""""""""""
+
+    Used by: :ref:`TcaIndexer`.
+
+    Defines a blacklist of page uids. Records below any of these pages, or subpages, are not
+    indexed. This allows you to define areas that should not be indexed.
+    The page attribute *No Search* is also taken into account to prevent indexing records from only one
+    page without recursion.
+
+    Contains a comma separated list of table names. Spaces are trimmed.
+
+    Example::
+
+        plugin.tx_searchcore.settings.index.rootLineBlacklist = 3, 10, 100
+
+Also it's possible to define some behaviour for the different document types. In context of TYPO3
+tables are used as document types 1:1. It's possible to configure different tables. The following
+options are available:
+
+.. _additionalWhereClause:
+
+``additionalWhereClause``
+"""""""""""""""""""""""""
+
+    Used by: :ref:`TcaIndexer`.
+
+    Add additional SQL to where clauses to determine indexable records from the table. This way you
+    can exclude specific records like ``tt_content`` records with specific ``CType`` values or
+    something else. E.g. you can add a new field to the table to exclude records from indexing.
+
+    Example::
+
+        plugin.tx_searchcore.settings.index.tt_content.additionalWhereClause = tt_content.CType NOT IN ('gridelements_pi1', 'list', 'div', 'menu')
+
+    .. attention::
+
+        Make sure to prefix all fields with the corresponding table name. The selection from
+        database will contain joins and can lead to SQL errors if a field exists in multiple tables.

Documentation/source/connections.rst

@@ -0,0 +1,26 @@
+.. _connections:
+
+Connections
+===========
+
+See Concept of :ref:`concepts_connections` for further background information.
+
+The extension provides the following connections out of the box:
+
+.. _Elasticsearch:
+
+Elasticsearch
+-------------
+
+Integrates `elastic Elasticsearch`_ using `elastica`_ into TYPO3.
+
+Provides basic support like indexing without mappings and full text search at the moment.
+
+The connection is configurable through the following options:
+
+* :ref:`host`
+
+* :ref:`port`
+
+.. _elastic Elasticsearch: https://www.elastic.co/products/elasticsearch
+.. _elastica: http://elastica.io/

Documentation/source/index.rst

@@ -33,6 +33,12 @@ Table of Contents
 =================
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
+   :glob:
 
    installation
+   configuration
+   usage
+   concepts
+   connections
+   indexer

Documentation/source/indexer.rst

@@ -0,0 +1,28 @@
+.. _indexer:
+
+Indexer
+=======
+
+See Concept of :ref:`concepts_indexing` for further background information.
+
+The extension provides the following indexer out of the box:
+
+.. _TcaIndexer:
+
+TcaIndexer
+----------
+
+Provides zero configuration TYPO3 integration by using the :ref:`t3-tca-ref:start`. You just can
+start indexing TYPO3.
+
+The indexer will use the TCA to fetch all necessary information like relations. Currently the
+implementation is very basic. In future it will also provide mapping for Elasticsearch and further
+stuff.
+
+The indexer is configurable through the following options:
+
+* :ref:`allowedTables`
+
+* :ref:`rootLineBlacklist`
+
+* :ref:`additionalWhereClause`

Documentation/source/installation.rst

@@ -1,45 +1,16 @@
-.. heighlight:: typoscript
+.. highlight:: bash
 .. _installation:
 
 Installation
 ============
 
-The extension can be installed through composer:
+The extension can be installed through composer::
 
-.. code-block:: bash
+    composer require "leonmrni/search_core dev-feature/integrate-elasticsearch"
 
-    composer require "danielsiepmann/search_core" "dev-feature/integrate-elasticsearch"
+or by `downloading`_ and placing it inside the :file:`typo3conf/ext`-Folder of your installation.
-
-or by downloading and placing it inside the :file:`typo3conf/ext`-Folder of your installation.
 
 Afterwards you need to enable the extension through the extension manager and include the static
 typoscript setup.
 
-.. _configuration:
+.. _downloading: https://github.com/DanielSiepmann/search_core/archive/feature/integrate-elasticsearch.zip
-
-Configuration
-=============
-
-The extension offers the following configuration options through TypoScript. If you overwrite them
-through `setup` make sure to keep them in the `module` area as they will be accessed from backend
-mode of TYPO3. Do so by placing the following line at the end::
-
-    module.tx_searchcore < plugin.tx_searchcore
-
-.. todo::
-
-   We will use references inside the extension to make the above unnecessary in the future.
-
-Currently no constants are available, but this will change in the near future to make configuration
-easier.
-
-The strucutre is following TYPO3 Extbase conventions. All settings are placed inside of::
-
-    plugin.tx_searchcore.settings
-
-Here is the example default configuration that's provided through static setup:
-
-.. literalinclude:: ../../Configuration/TypoScript/setup.txt
-   :language: typoscript
-   :linenos:
-   :caption: Static TypoScript Setup

Documentation/source/usage.rst

@@ -0,0 +1,23 @@
+.. highlight:: bash
+.. _usage:
+
+Usage
+=====
+
+.. _usage_manual_indexing:
+
+Manual indexing
+---------------
+
+You can trigger indexing from CLI::
+
+    ./typo3/cli_dispatch.phpsh extbase index:index --table 'tt_content'
+
+This will index the table ``tt_content`` using the :ref:`TcaIndexer`.
+
+.. _usage_auto_indexing:
+
+Auto indexing
+-------------
+
+Indexing is done through hooks everytime an record is changed.