Creating Major Release Notes

The major release notes attempt to document every user-visible and user-impactful change. Performance improvements are not mentioned unless they are user-visible (e.g., new syntax) or significant enough to enable new workloads. Similarly, while the major release notes include changes to the documented extension interface, it does not include all changes of interest to extension developers or Postgres forks because doing so would include too many items that would be uninteresting to the general audience.

0. Find branch date with query

       git show --summary $(git merge-base REL_18_STABLE master)

1. 01raw ("01raw" is the file name I use)

       read src/tools/git_changelog
       run src/tools/git_changelog
       src/tools/git_changelog --master-only --brief --oldest-first --since='start-date'

2. 02trim

       add "[DOC" text to indicate commits that changed documentation using:

awk -F'[\\[\\]]' '
        BEGIN {doc = "";}
        {
                if ($0 ~ /^20[[:digit:]]{2}-[[:digit:]]{2}-[[:digit:]]{2} \[[[:xdigit:]]+\] /)
                {
                        cmd = "git show --name-only --oneline " $2 " | wc -l";
                        cmd | getline lines;
                        close(cmd);

                        cmd = "git show --name-only --oneline " $2 " | grep -c \"^doc/\"";
                        cmd | getline doc_lines;
                        close(cmd);

                        if (doc_lines > 0)
                        {
                                # skip commit subject line
                                if (doc_lines < lines - 1)
                                        doc = "[DOC & CODE]";
                                else
                                        doc = "[DOC ONLY]";
                        }
                }

                # Put label on empty line so the first commit text is always in
                # the same place.
                if (doc != "" && $0 == "")
                {
                        print doc;
                        doc = "";
                }
                else
                        print $0;
        }' "$@"

       add "AS OF" date
       load relnote macro
               ESC-X, source pg.mac
       use text editor macro F7 to delete each commit
               F3/F4 for back/forward commits  |  F7 delete commit  |  F8 delete commit detail below cursor
               I use MicroEmacs 4 macros to do this.
       delete commit with cursor anywhere in the commit text
       check 01raw for missing items and incompatibilities

3. 03comments

       replace "[DOC " with a blank line
               sed 's/^\[DOC //'
       add comments around commit tags
       remove leading space on commit text

4. 04reword

       reword items

5. 05tags

       update for current git
       spellcheck
       remove <>&
               grep '^[^<-][^:]*[<>&]'
       remove trailing spaces
       remove double-blank lines
       add </listitem><listitem>
       add </para><para>
       fix </para><para for double-paras

6. 06groups

       copy previous major release notes to 06groups
       append 05tags
               cat 05tags >> 06groups
       Release date YYYY-??-??
       add "AS OF" date
       make groups using text editor
       remove text between <listitem>
       remove acknowledgement <member>
       chown file for gedit access

       use gedit text editor
               add line numbers in editor
               use Colbalt colors so current line is clearly highlighted
               display overview map
               ^G to go to last search, search for "-----"
               use triple-click to select whole lines
       move items into groups
       move items into incompatibilities

7. 07order

       order inside of sections
               most important item first in section
                  if none, then largest group first
               group similar items together
               order group by item importance
               ideally the last item in a group will relate to the first item in the next group

8. 08spell

       2nd pass of spellcheck
               remove HTML comments, authors
               cat 08spell | grep -Ev '^(20[12][0-9]-|Author: )|commit_baseurl;[0-9a-f]*' | sed 's/ (.\+//g' > /tmp/x;  ispell -H /tmp/x
       add space between double dashes in XML comments
               in text editor, search for '^202.*--'
       check contributor names for consistency
               grep '([^(]*)$' 08spell | sed 's/^[^(]*(//' | tr -d ')'| sed 's/,/\n/g' | sed 's/^[( ]\+//g' | sort -u > /tmp/first
       checck names spelled first/last and last/first
               awk '{print $1,$2; print $2,$1}' /tmp/first | sort | uniq -c | sort -nr | awk '$1>1' > /tmp/last
       install in doc/src/sgml
       run src/tools/add_commit_links.pl
       check PDF output for invalid characters

9. PDF

       run make postgres-US.pdf and look for non-output character warning

10. Post release note URL to pgsql-hackers@lists.postgresql.org

11. Post blog after a few days of feedback

12. To show commits that were excluded, use:

# Usage:  command git_changelog_output release_notes

# Create a "!=" string to compare all commit hashes in the release notes.
PATTERN=$(awk -F'[\\[\\]]' '
        BEGIN {first = "Y";}
        {
                if ($0 ~ /^20[[:digit:]]{2}-[[:digit:]]{2}-[[:digit:]]{2} \[[[:xdigit:]]+\] /)
                {
                        if (first != "Y")
                                printf(" && ");
                        first="N";
                        printf("$2 != \"" $2 "\"");
                }
        }' "$2"
)

awk -F'[\\[\\]]' '
        {
                if ($0 ~ /^Author: /)
                        save_author=$0;
                else if ($0 ~ /^20[[:digit:]]{2}-[[:digit:]]{2}-[[:digit:]]{2} \[[[:xdigit:]]+\] /)
                {
                        if ('"$PATTERN"')
                        {
                                print save_author;
                                print $0;
                        }
                        else
                                save_author="";
                }
                else if (save_author != "")
                        print $0;
        }' "$1"

and then run the script in #2 to add "[DOC" markers; email to pgsql-hackers@lists.postgresql.org.

13. Do the following after community feedbackscript, hopefully before beta1

       delayed because
               editing text with markup is hard
               finding text with markup is hard
               only the first mention gets a link, and reordering would mess that up
       add markup, see doc/src/sgml/release.sgml
       add links by tag
               add only to first mention in adjacent item list
               get tag list from release.sgml
               use "<xref" for varname, applications
               use "<xref" for command where text is only the command name
                       the rest need "<link linkend"
               use "<link linkend" for struct
               go over all items for general links
               check invalid for "section", "chapter", and "???" text

14. Do this near the final release date so diff analysis is easier

       use pipe to mark commit paras
       indentation
               <sect4 is indented by four spaces
               use pipe to prefix <ulink> & </para>
       indent itemizedlist
       use ESC-tab to indent paras and word wrap
       outdent para tags
       word wrap
       recheck contributor names for consistency
       update TODO list