Commit f6d01b8b authored by Paul Sokolovsky's avatar Paul Sokolovsky
Browse files

docs: Add sphinx_selective_exclude extension suite.

Designed specifically to workaround issues we were facing with generating
multiple conditionalized output docsets from a single master doctree.
Extensions were factored out into a separate project, based on the fact
that many other Sphinx users experience similar or related problems:
https://github.com/pfalcon/sphinx_selective_exclude

Corresponds to the 182f4a8da57 upstream revision.
parent 9de5eb27
Copyright (c) 2016 by the sphinx_selective_exclude authors.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Sphinx eager ".. only::" directive and other selective rendition extensions
===========================================================================
Project home page: https://github.com/pfalcon/sphinx_selective_exclude
The implementation of ".. only::" directive in Sphinx documentation
generation tool is known to violate principles of least user surprise
and user expectations in general. Instead of excluding content early
in the pipeline (pre-processor style), Sphinx defers exclusion until
output phase, and what's the worst, various stages processing ignore
"only" blocks and their exclusion status, so they may leak unexpected
information into ToC, indexes, etc.
There's multiple issues submitted upstream on this matter:
* https://github.com/sphinx-doc/sphinx/issues/2150
* https://github.com/sphinx-doc/sphinx/issues/1717
* https://github.com/sphinx-doc/sphinx/issues/1488
* etc.
They are largely ignored by Sphinx maintainers.
This projects tries to rectify situation on users' side. It actually
changes the way Sphinx processes "only" directive, but does this
without forking the project, and instead is made as a standard
Sphinx extension, which a user may add to their documentation config.
Unlike normal extensions, extensions provided in this package
monkey-patch Sphinx core to work in a way expected by users.
eager_only
----------
The core extension provided by the package is called `eager_only` and
is based on the idea by Andrea Cassioli (see bugreports above) to
process "only" directive as soon as possible during parsing phase.
This approach has some drawbacks, like producing warnings like
"WARNING: document isn't included in any toctree" if "only" is used
to shape up a toctree, or the fact that changing a documentation
builder (html/latex/etc.) will almost certainly require complete
rebuild of documentation. But these are relatively minor issues
comparing to completely broken way "only" works in upstream Sphinx.
modindex_exclude
----------------
"only" directive allows for fine-grained conditional exclusion, but
sometimes you may want to exclude entire module(s) at once. Even if
you wrap an entire module description in "only" directive, like:
.. only: option1
.. module:: my_module
...
You will still have an HTML page generated, albeit empty. It may also
go into indexes, so will be discoverable by users, leading to less
than ideal experience. `modindex_exclude` extension is design to
resolve this issue, by making sure that any reference of a module
is excluded from Python module index ("modindex"), as well as
general cross-reference index ("genindex"). In the latter case,
any symbol belong to a module will be excluded. Unlike `eager_only`
extension which appear to have issued with "latexpdf" builder,
`modindex_exclude` is useful for PDF, and allows to get cleaner
index for PDF, just the same as for HTML.
search_auto_exclude
-------------------
Even if you exclude soem documents from toctree:: using only::
directive, they will be indexed for full-text search, so user may
find them and get confused. This plugin follows very simple idea
that if you didn't include some documents in the toctree, then
you didn't want them to be accessible (e.g. for a particular
configuration), and so will make sure they aren't indexed either.
This extension depends on `eager_only` and won't work without it.
Note that Sphinx will issue warnings, as usual, for any documents
not included in a toctree. This is considered a feature, and gives
you a chance to check that document exclusions are indeed right
for a particular configuration you build (and not that you forgot
to add something to a toctree).
Summary
-------
Based on the above, sphinx_selective_exclude offers extension to let
you:
* Make "only::" directive work in an expected, intuitive manner, using
`eager_only` extension.
* However, if you apply only:: to toctree::, excluded documents will
still be available via full-text search, so you need to use
`search_auto_exclude` for that to work as expected.
* Similar to search, indexes may also require special treatment, hence
there's the `modindex_exclude` extension.
Most likely, you will want to use all 3 extensions together - if you
really want build subsets of docimentation covering sufficiently different
configurations from a single doctree. However, if one of them is enough
to cover your usecase, that's OK to (and why they were separated into
3 extensions, to follow KISS and "least surprise" principles and to
not make people deal with things they aren't interested in). In this case,
however remember there're other extensions, if you later hit a usecase
when they're needed.
Usage
-----
To use these extensions, add https://github.com/pfalcon/sphinx_selective_exclude
as a git submodule to your project, in documentation folder (where
Sphinx conf.py is located). Alternatively, commit sphinx_selective_exclude
directory instead of making it a submodule (you will need to pick up
any project updates manually then).
Add following lines to "extensions" settings in your conf.py (you
likely already have some standard Sphinx extensions enabled):
extensions = [
...
'sphinx_selective_exclude.eager_only',
'sphinx_selective_exclude.search_auto_exclude',
'sphinx_selective_exclude.modindex_exclude',
]
As discussed above, you may enable all extensions, or one by one.
Please note that to make sure these extensions work well and avoid producing
output docs with artifacts, it is IMPERATIVE to remove cached doctree if
you rebuild documentation with another builder (i.e. with different output
format). Also, to stay on safe side, it's recommended to remove old doctree
anyway before generating production-ready documentation for publishing. To
do that, run something like:
rm -rf _build/doctrees/
A typical artificat when not following these simple rules is that content
of some sections may be missing. If you face anything like that, just
remember what's written above and remove cached doctrees.
#
# This is a Sphinx documentation tool extension which makes .only::
# directives be eagerly processed early in the parsing stage. This
# makes sure that content in .only:: blocks gets actually excluded
# as a typical user expects, instead of bits of information in
# these blocks leaking to documentation in various ways (e.g.,
# indexes containing entries for functions which are actually in
# .only:: blocks and thus excluded from documentation, etc.)
# Note that with this extension, you may need to completely
# rebuild a doctree when switching builders (i.e. completely
# remove _build/doctree dir between generation of HTML vs PDF
# documentation).
#
# This extension works by monkey-patching Sphinx core, so potentially
# may not work with untested Sphinx versions. It tested to work with
# 1.2.2 and 1.4.2
#
# Copyright (c) 2016 Paul Sokolovsky
# Based on idea by Andrea Cassioli:
# https://github.com/sphinx-doc/sphinx/issues/2150#issuecomment-171912290
# Licensed under the terms of BSD license, see LICENSE file.
#
import sphinx
from docutils.parsers.rst import directives
class EagerOnly(sphinx.directives.other.Only):
def run(self, *args):
# Evaluate the condition eagerly, and if false return no nodes right away
env = self.state.document.settings.env
env.app.builder.tags.add('TRUE')
#print(repr(self.arguments[0]))
if not env.app.builder.tags.eval_condition(self.arguments[0]):
return []
# Otherwise, do the usual processing
nodes = super(EagerOnly, self).run()
if len(nodes) == 1:
nodes[0]['expr'] = 'TRUE'
return nodes
def setup(app):
directives.register_directive('only', EagerOnly)
#
# This is a Sphinx documentation tool extension which allows to
# exclude some Python modules from the generated indexes. Modules
# are excluded both from "modindex" and "genindex" index tables
# (in the latter case, all members of a module are exlcuded).
# To control exclusion, set "modindex_exclude" variable in Sphinx
# conf.py to the list of modules to exclude. Note: these should be
# modules (as defined by py:module directive, not just raw filenames).
# This extension works by monkey-patching Sphinx core, so potentially
# may not work with untested Sphinx versions. It tested to work with
# 1.2.2 and 1.4.2
#
# Copyright (c) 2016 Paul Sokolovsky
# Licensed under the terms of BSD license, see LICENSE file.
#
import sphinx
#org_PythonModuleIndex_generate = None
org_PyObject_add_target_and_index = None
org_PyModule_run = None
EXCLUDES = {}
# No longer used, PyModule_run() monkey-patch does all the job
def PythonModuleIndex_generate(self, docnames=None):
docnames = []
excludes = self.domain.env.config['modindex_exclude']
for modname, (docname, synopsis, platforms, deprecated) in self.domain.data['modules'].items():
#print(docname)
if modname not in excludes:
docnames.append(docname)
return org_PythonModuleIndex_generate(self, docnames)
def PyObject_add_target_and_index(self, name_cls, sig, signode):
if hasattr(self.env, "ref_context"):
# Sphinx 1.4
ref_context = self.env.ref_context
else:
# Sphinx 1.2
ref_context = self.env.temp_data
modname = self.options.get(
'module', ref_context.get('py:module'))
#print("*", modname, name_cls)
if modname in self.env.config['modindex_exclude']:
return None
return org_PyObject_add_target_and_index(self, name_cls, sig, signode)
def PyModule_run(self):
env = self.state.document.settings.env
modname = self.arguments[0].strip()
excl = env.config['modindex_exclude']
if modname in excl:
self.options['noindex'] = True
EXCLUDES.setdefault(modname, []).append(env.docname)
return org_PyModule_run(self)
def setup(app):
app.add_config_value('modindex_exclude', [], 'html')
# global org_PythonModuleIndex_generate
# org_PythonModuleIndex_generate = sphinx.domains.python.PythonModuleIndex.generate
# sphinx.domains.python.PythonModuleIndex.generate = PythonModuleIndex_generate
global org_PyObject_add_target_and_index
org_PyObject_add_target_and_index = sphinx.domains.python.PyObject.add_target_and_index
sphinx.domains.python.PyObject.add_target_and_index = PyObject_add_target_and_index
global org_PyModule_run
org_PyModule_run = sphinx.domains.python.PyModule.run
sphinx.domains.python.PyModule.run = PyModule_run
#
# This is a Sphinx documentation tool extension which allows to
# automatically exclude from full-text search index document
# which are not referenced via toctree::. It's intended to be
# used with toctrees conditional on only:: directive, with the
# idea being that if you didn't include it in the ToC, you don't
# want the docs being findable by search either (for example,
# because these docs contain information not pertinent to a
# particular product configuration).
#
# This extension depends on "eager_only" extension and won't work
# without it.
#
# Copyright (c) 2016 Paul Sokolovsky
# Licensed under the terms of BSD license, see LICENSE file.
#
import sphinx
org_StandaloneHTMLBuilder_index_page = None
def StandaloneHTMLBuilder_index_page(self, pagename, doctree, title):
if pagename not in self.env.files_to_rebuild:
if pagename != self.env.config.master_doc and 'orphan' not in self.env.metadata[pagename]:
print("Excluding %s from full-text index because it's not referenced in ToC" % pagename)
return
return org_StandaloneHTMLBuilder_index_page(self, pagename, doctree, title)
def setup(app):
global org_StandaloneHTMLBuilder_index_page
org_StandaloneHTMLBuilder_index_page = sphinx.builders.html.StandaloneHTMLBuilder.index_page
sphinx.builders.html.StandaloneHTMLBuilder.index_page = StandaloneHTMLBuilder_index_page
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment