summaryrefslogtreecommitdiff
path: root/tools/docs/docload.py
AgeCommit message (Collapse)Author
2022-06-20Fix typo in commentMagnus Hagander
2022-06-20Refactor docsload and add support for loading from directoryMagnus Hagander
Break out the tar parsing parts into a function for just that and create a new one that knows how to load directly from the html directory in the postgresql source tree, for more efficient snapshot loading.
2022-06-20Store the git hash of developer docs loadedMagnus Hagander
This will require some further updates on the loading side of things before it's fully valid, but for now track and show a link to the git hash used to build developer docs *if* one is specified. We only track it for devel (because releases have release numbers) and we only show it in the cases where we would already show the loading time.
2022-06-20Use the proper set of arguments in docs loaderMagnus Hagander
We parsed it out using the options parser, but then when using it required a hardcoded order on the commandline. That made no sense...
2021-08-11Add verbose mode to docs loading and tweak defaultsMagnus Hagander
Instead of printing every single page loaded, print the start of the process and the statitics by default. The existing --quiet parameter continues to work to make the process completely quiet. Add a new parameter --verbose that makes it run in the old way, printing everything.
2021-05-18Make documentation purges use xkeyMagnus Hagander
2020-11-23Only flag docs as loaded when they've actually changedMagnus Hagander
We already avoided the actual updates (by filtering the UPDATE statement), but since we set the load date we'd trigger a change to every page and kick it ouf of the caches even when not chagned, which is wasteful. So instead only do that if something has changed. When it has, we still reset that whole version of the docs since we want the load date to be consistent across them.
2020-01-09Fix a number of incorrect escapesMagnus Hagander
Pointed out by newer versions of pep8, but they were never correct.
2019-04-25Add Bootstrap grid classes to images referenced in the HTML docs.Jonathan S. Katz
2019-03-31Add a lot more code documentation to the documentation loader.Jonathan S. Katz
2019-03-31Do not use libtidy on content from SVG files.Jonathan S. Katz
2019-03-31Remove global variable reference that increments the page count in the ↵Jonathan S. Katz
docloader. This is primarily to remove some old code to allow for future changes to how certain types of doc files can be loaded in the future.
2019-02-22Load docs using COPY instead of INSERTMagnus Hagander
Issuing individual INSERTs for each line in the docs works decently when local, but it slow when loading across The Tubes Of The Internet. Switching to using COPY takes the load time from the buildfarm animal from just over 2 minutes to about 6-7 seconds.
2019-01-26Update docsload to use tidylib instead of tidyMagnus Hagander
The old library is not supported on py3. This is the same migration we've done for other projects like pgarchives.
2019-01-26More generic changes from 2to3Magnus Hagander
2019-01-26Update script shebangs to use python3Magnus Hagander
2019-01-26Update print and input syntax for python 3Magnus Hagander
2019-01-26Generic unicode updatesMagnus Hagander
2019-01-17Remove multi-statement lines, per pep8Magnus Hagander
2019-01-17Fix whitespace and indentation, per pep8Magnus Hagander
2019-01-17Tabs, meet your new overlords: spacesMagnus Hagander
In a quest to reach pep8, use spaces to indent rather than tabs.
2018-04-03Turn off tidy indent on docs older than 11Magnus Hagander
It seems the newer tidy in Debian Stretch breaks with the output from the old docs toolchain, causing indention to happen inside <pre> blocks which clearly breaks rendering. Turn it off for thos, but keep it enabled for version 11 and up (at this point that's just dev), because the output becomes a lot easier to read when trying to debug things.
2017-02-18Disable clean flag in tidy calls on new XML based docs buildMagnus Hagander
Let's see if this cleans up some unintentional formatting
2016-11-16Support loading docs in utf-8 from pg version 10Magnus Hagander
2016-01-29Go back to using direct psycopg2 for docs loadingMagnus Hagander
This removes the dependency on django from docload, facilitating incremental upgrades of the infrastructure. This now requires a new docload.ini file in the tools/docs directory, with a section "db" and a setting "dsn".
2012-09-24When loading the current version of docs, also purge /current/Magnus Hagander
2012-05-18Close db connection in docloaderMagnus Hagander
To make Stefan happy...
2011-12-04Include docs load timestamp in developer docsMagnus Hagander
This makes it possible to figure out when the docs were actually loaded, since developer docs don't carry a version number. This is actually going to be the docs *load* timestamp, and not build timestamp, but they should be close enough together that it shouldn't matter. Fixes #108
2011-12-04Purge the correct URL when loading developer docsMagnus Hagander
They're /docs/devel/, not /docs/0/ - that's just used to cheat the version system being a decimal field...
2011-11-27Ignore symlinks in docs tarballsMagnus Hagander
Required for postgresql <7.1
2011-11-23Purge newly loaded docs from varnish automaticallyMagnus Hagander
2011-11-22Add option to docload.py to make it run quietlyMagnus Hagander
Using commandline --quiet. To do this, switch to using the standard python optionparser for commandline parsing.
2011-08-16Make docs loader use the django database connection/configurationMagnus Hagander
In passing, make it check that the version being loaded actually exists...
2009-09-14A first very basic import.Magnus Hagander
Contains basic functionality, and an import of most of the static content from the old site. There is still plenty more to do...
generated by cgit v1.2.3 (git 2.39.1) at 2025-07-18 11:28:43 +0000