MediaWiki API result

This is the HTML representation of the JSON format. HTML is good for debugging, but is unsuitable for application use.

Specify the format parameter to change the output format. To see the non-HTML representation of the JSON format, set format=json.

See the complete documentation, or the API help for more information.

{
    "batchcomplete": "",
    "continue": {
        "gapcontinue": "Refactor_Type_System",
        "continue": "gapcontinue||"
    },
    "warnings": {
        "main": {
            "*": "Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes."
        },
        "revisions": {
            "*": "Because \"rvslots\" was not specified, a legacy format has been used for the output. This format is deprecated, and in the future the new format will always be used."
        }
    },
    "query": {
        "pages": {
            "3875": {
                "pageid": 3875,
                "ns": 0,
                "title": "Readahead",
                "revisions": [
                    {
                        "contentformat": "text/x-wiki",
                        "contentmodel": "wikitext",
                        "*": "\n= in Linux =\n\n== Summary ==\n\n; Very concise summary\n: readahead does read.\n\n;Concise summary\n: readahead in linux is more similar to readv implementation in PostgreSQL (at least the stream IO implemented by Thomas Munro for pg17).\n: It might be easier to read it this way to understand how it works.\n\n;Important points\n: readahead is not async\n: readahead is soon filesystem specific, see bottom of call tree below.\n\nOn linux, readahead as evolved over time and the last known review has been done by Neil Brown around April 8, 2022.<br/>\nHe wrote an article to summarize what he discovered: [https://lwn.net/Articles/888715/ Readahead: the documentation I wanted to read].\n\nWhich lead to this updated doc (apparently merge in 5.18): https://www.kernel.org/doc/html/latest/core-api/mm-api.html#readahead\n\nThis is indeed a very nice summary and it covers mostly 2 entry functions for readahead: ''page_cache_sync_ra()'' and ''page_cache_async_ra()''.\n\nBoth have a nice up to date documentation about their underlying functions: https://www.kernel.org/doc/html/latest/core-api/mm-api.html?highlight=page_cache_async_readahead#c.page_cache_sync_readahead\n\n----\n\nTake a moment to read the documentation (excerpt here):\n\n; page_cache_sync_readahead()\n: should be called when a cache miss happened: '''it will submit the read'''. The readahead logic may decide to piggyback more pages onto the read request if access patterns suggest it will improve performance.\n\n; page_cache_async_readahead()\n: should be called when a page is used which is marked as PageReadahead; this is a marker to suggest that the application has used up enough of the readahead window that we should start pulling in more pages.</q>\n\nOK, so far so good.\n\nBut the code when doing ''posix_fadvise'' is distinct, and the entry point lead to:\n\n; page_cache_ra_unbounded()\n: This function is for filesystems to call when they want to start readahead beyond a file's stated ''i_size''. This is almost certainly not the function you want to call. Use ''page_cache_async_readahead()'' or ''page_cache_sync_readahead()'' instead. File is referenced by caller. Mutexes may be held by caller. '''May sleep, but will not reenter filesystem to reclaim memory.'''\n\nToday, linux is using \"folios\", and the readahead flag is set via ''folio_set_readahead(folio);'' in this function. \n\nOther super important part is: \"can wait\", yes IT IS NOT ASYNCHRONOUS. The only situation where it's partly asynchronous is when the storage is really congested and so during the execution of the call on the provided readahead range, the process is aborted. As mentioned by Neil the \"congestion\" has not been followed everywhere and may not work as expected.\n\nSide note: linux split range in chunks of 2MB to manage memory and reduce locking. Hardcoded.\n\nAn excerpt from the comment in the code of [https://elixir.bootlin.com/linux/latest/source/mm/readahead.c#L27 readahead]: \n\n    /*\n     * Each readahead request is partly synchronous read, and partly async\n     * readahead.  This is reflected in the struct file_ra_state which\n     * contains ->size being the total number of pages, and ->async_size\n     * which is the number of pages in the async section.  The readahead\n     * flag will be set on the first folio in this async section to trigger\n     * a subsequent readahead.  Once a series of sequential reads has been\n     * established, there should be no need for a synchronous component and\n     * all readahead request will be fully asynchronous.\n     */\n\n== Call tree - The functions really used by linux posix_fadvise ==\n\nhttps://elixir.bootlin.com/linux/latest/source/mm/fadvise.c#L31\n\n    /*\n    * POSIX_FADV_WILLNEED could set PG_Referenced, and POSIX_FADV_NOREUSE could\n    * deactivate the pages and clear PG_Referenced.\n    */\n    int generic_fadvise(struct file *file, loff_t offset, loff_t len, int advice)\n\nhttps://elixir.bootlin.com/linux/latest/source/mm/internal.h#L126\n\n    inline wrapper\n\nhttps://elixir.bootlin.com/linux/latest/source/mm/readahead.c#L306\n\n    /*\n    * Chunk the readahead into 2 megabyte units, so that we don't pin too much\n    * memory at once.\n    */\n    void force_page_cache_ra(struct readahead_control *ractl,\n            unsigned long nr_to_read)\n\t\t\nhttps://elixir.bootlin.com/linux/latest/source/mm/readahead.c#L281\n\n    /*\n    * do_page_cache_ra() actually reads a chunk of disk.  It allocates\n    * the pages first, then submits them for I/O. This avoids the very bad\n    * behaviour which would occur if page allocations are causing VM writeback.\n    * We really don't want to intermingle reads and writes like that.\n    */\n    \nhttps://elixir.bootlin.com/linux/latest/source/mm/readahead.c#L205\n\n    /**\n    * page_cache_ra_unbounded - Start unchecked readahead.\n    * @ractl: Readahead control.\n    * @nr_to_read: The number of pages to read.\n    * @lookahead_size: Where to start the next readahead.\n    *\n    * This function is for filesystems to call when they want to start\n    * readahead beyond a file's stated i_size.  This is almost certainly\n    * not the function you want to call.  Use page_cache_async_readahead()\n    * or page_cache_sync_readahead() instead.\n    *\n    * Context: File is referenced by caller.  Mutexes may be held by caller.\n    * May sleep, but will not reenter filesystem to reclaim memory.\n    */ \n\nsome interesting comments in it, during preallocation:\n\n    /*\n     * Partway through the readahead operation, we will have added\n      * locked pages to the page cache, but will not yet have submitted\n      * them for I/O.  Adding another page may need to allocate memory,\n      * which can trigger memory reclaim.  Telling the VM we're in\n      * the middle of a filesystem operation will cause it to not\n      * touch file-backed pages, preventing a deadlock.  Most (all?)\n      * filesystems already specify __GFP_NOFS in their mapping's\n      * gfp_mask, but let's be explicit here.\n      */\n \nhttps://elixir.bootlin.com/linux/latest/source/mm/readahead.c#L146\n\n    static void read_pages(struct readahead_control *rac)\n\nThen it is filesystem specific.\n\n=== EXT4 ===\n\nhttps://elixir.bootlin.com/linux/latest/source/fs/ext4/inode.c#L3124\n\n    static int ext4_read_folio(struct file *file, struct folio *folio)\n\nif not found in mem:\n\nhttps://elixir.bootlin.com/linux/latest/source/fs/ext4/readpage.c#L211\n\n    int ext4_mpage_readpages(struct inode *inode,\n            struct readahead_control *rac, struct folio *folio)\n\nafter lot of logic around sequential, hole, ...:\n\nhttps://elixir.bootlin.com/linux/latest/source/block/blk-core.c#L833\n\n\n    /**\n     * submit_bio - submit a bio to the block device layer for I/O\n     * @bio: The &struct bio which describes the I/O\n     *\n     * submit_bio() is used to submit I/O requests to block devices.  It is passed a\n     * fully set up &struct bio that describes the I/O that needs to be done.  The\n     * bio will be send to the device described by the bi_bdev field.\n     *\n     * The success/failure status of the request, along with notification of\n     * completion, is delivered asynchronously through the ->bi_end_io() callback\n     * in @bio.  The bio must NOT be touched by the caller until ->bi_end_io() has\n     * been called.\n     */\n\n== First thoughts ==\n\n* having a WILLNEED on a sequential pattern just competes with linux own ra.\n\n* if PG_readahead is set, linux will interpret that as a successful past readahead and may keep on doing more ra from this block when reading (it's mitigated by some hole detection apparently).\n* if PG_readahead is not set when reading, not checked how the ra logic will use that.\n\n* having a DONTNEED just after a read is apparently optimized in linux code.\n\n* setting RANDOM or SEQUENTIAL flag influence the linux default ra effectively. And is costless.\n\n\n[[Category:development]]"
                    }
                ]
            },
            "190": {
                "pageid": 190,
                "ns": 0,
                "title": "ReadOnlyTables",
                "revisions": [
                    {
                        "contentformat": "text/x-wiki",
                        "contentmodel": "wikitext",
                        "*": "{{CategoryBox}}\n\nv0.2 12 Dec 2007\n\nPostgres supports the concept of freezing tuples, so they can live\nforever within the database without needing further writes. Currently\nthere is no command that will guarantee that a table has been completely\nfrozen. This makes it difficult to reliably write data files to WORM\nmedia for longer term archiving. (WORM means Write-Once, Read-Many).\nIt's also a pain having to VACUUM a large table again just because a\nsmall number of rows need to be frozen.\n\nSo we need a DDL command that will ensure all tuples are frozen and then\nmark the table as read-only. Ideally, we would like to do this in a way\nthat doesn't hold long full table locks, since we want the data to\nremain accessible at all times.\n\nSo... VACUUM FREEZE table SET READ ONLY;\n\nwould be my first thought, but I'm guessing everybody will press me\ntowards supporting the more obvious\n\nALTER TABLE table SET READ ONLY;\n\nThis command will place a ShareLock (only) on the table, preventing\nanybody from writing to the table while we freeze it. The ShareLock is\nincompatible with any transaction that has written to the table, so when\nwe acquire the lock all writers to the table will have completed. We\nthen run the equivalent of a VACUUM FREEZE which will then be able to\nfreeze *all* rows in one pass (rather than all except the most recent).\n\nThe freeze pass will progress until it sees a potentially invisible row,\ni.e. one that has been written by a recent TransactionId.\nAt that point the ALTER TABLE will wait for that TransactionId to \ncomplete, before continuing to progress. We assume that there are few\nrows that will be potentially invisible, so we don't bother to update\nOldestXmin as we progress through the table. So a table recently updated\nby a long running transaction could cause the ALTER TABLE to wait for\nan extended period, much longer than the time for the table scan.\n\nOn completion of the freeze pass we will then update the pg_class entry\nto show that it is now read-only, so we will emulate the way VACUUM does\nthis.\n\nThis form of the ALTER TABLE command will need to be mangled so it can\nonly run outside of a transaction block and also so it takes only a\nShareLock rather than an AccessExclusiveLock.\n\nReversing the process is simpler, since we only have to turn off the\nflag in pg_class:\n\nALTER TABLE table SET READ WRITE;\n\npossibly able to do this without grabbing an AccessExclusiveLock, though\nthat isn't an important part of this implementation.\n\nRead-only tables never need VACUUMing, so we would be able to make\nautovacuum and explicit vacuum ignore them.\n\nRead-only tables may not be written to, yet would still allow implicit\nor explicit INSERT, UPDATE and DELETE privileges to be held on the\ntable. Attempts to write to the table will result in a specific \"read\nonly table cannot be modified\" ERROR. This allows a table to be placed\ninto read-only mode for long periods of time and flipped back to\nread-write if some updates are required. That is useful for various\nperformance reasons, see later. We can't use the privilege mechanism to\nprevent writes since superusers bypass them. (Thoughts?)\n\nRequests for tuple-level SHARE locks (e.g. FOR SHARE) against read-only\ntables will be ignored, since they are effectively already there. So we\ndon't need to change the internals of the locking, nor edit the RI code\nto remove the call to SHARE lock referenced tables. Do this during\npost-parse analysis.\n\nTables can be copied to WORM media by using\n\nALTER TABLE table SET TABLESPACE tblspc;\n\nThis would also use a ShareLock rather than an AccessExclusiveLock,\npiggy-backing off the work mentioned above.\n\nRunning SET TABLESPACE and SET READ ONLY at the same time might sound\nlike a good plan, but ISTM will require two fairly different code paths,\nso if we do it at all it will be a later addition.\n\n[[Category:Historical information]]"
                    }
                ]
            }
        }
    }
}