SCRAM authentication

Patch set for SCRAM authentication is currently being worked on at https://github.com/michaelpq/postgres/tree/scram.

Join the discussion on pgsql-hackers: https://www.postgresql.org/message-id/55192AFE.6080106%40iki.fi

TODO:

• Use SASLPrep for passwords (phases 1 and 2 of operation are complete, recomposition not yet).
  • [DONE] Per the spec NFKC needs to be used. charlint is an example of implementation in perl.
  • [DONE] In libpq, we need to check if the string is valid utf-8. If that's valid utf-8, apply SASLprep. if not, copy the string as-is. We could error as well in this case... Perhaps a WARNING could be more adapted, that's the most tricky case, and if the client does not use utf-8 that may lead to unexpected behavior.
  • [DONE] In server, when the password verifier is created. If client_encoding is utf-8, but not server_encoding, convert the password to utf-8 and build the verifier after applying SASLprep.
  • [DONE] UnicodeData.txt provides all the data that is needed for decomposition of strings. That's the 6th column of the data set.
  • [DONE] For Hangul characters, mathematical rules could be applied to get a clear decomposition.
  • [DONE] 1st phase of the conversion is the decomposition of the string using the mapping table.
  • [DONE] 2nd phase consists in applying combining classes and reorder the characters.
  • 3rd phase is a recomposition of the characters.

• [DONE] Check nonces, etc. to not contain invalid characters.

• [DONE] Derive mock SCRAM verifier for non-existent users deterministically from username.

• [DONE] Allow SCRAM authentication for users with a plain verifier in rolpassword.

• [DONE] Throw an error if an "authorization identity" is given. ATM, we just ignore it, but seems better to reject the attempt than do something that might not be what the client expects.

• [DONE] Add "scram-sha-256" prefix to SCRAM verifiers stored in pg_authid.rolpassword.

• [DONE] When attempting to connect with SCRAM, the logs complain but the connection is successful, this seems to be the result of a recent rebase:

FATAL:  password authentication failed for user "mpaquier"
DETAIL:  Connection matched pg_hba.conf line 84: "local   all             all                                     scram"
LOG:  could not send data to client: Broken pipe

• The message for incorrect password with md5 or password authentication is:

Password: 
psql: FATAL:  password authentication failed for user "heikki"

And for SCRAM:

Password: 
psql: error received from server in SASL exchange: invalid-proof

The md5 message is much nicer. It comes from the server, while for SCRAM, libpq prints out that. Can we make that nicer and/or more consistent?

• [DONE] libpq still sends the username in the SCRAM exchange. The plan was to send an empty string, meaning "use the username from startup packet". It fails currently if username contains a comma.
  • Per the spec, characters '=' and ',' would need to be changed. So with the username from the startup packet and an empty user name things should work properly:

The characters ',' or '=' in usernames are sent as '=2C' and
'=3D' respectively.  If the server receives a username that
contains '=' not followed by either '2C' or '3D', then the
server MUST fail the authentication.

• [DONE] commit 274bb2b3 (libpq: Allow connection strings and URIs to specify multiple hosts) changed the way password is found in pg_fe_sendauth() for MD5 and password authentication. I think we need the same changes for SCRAM

• [DONE] Extend initdb to support --auth=scram. Needs to tweak value of password_encryption in postgresql.conf in this case.

• Allow plain "password" authentication with a SCRAM verifier in pg_authid.rolpassword.

• (Give up on that?) Add "plain:" prefix to plaintext passwords stored in pg_authid.rolpassword.

• Improve OOM error in scram_Normalize() when checking for malloc() checks but this means having a pre-allocated error string to keep around within many nested calls.

• [DONE] Regression tests of plpython crash with the patches applied.

• [DONE] Make pg_hba.conf more extensible. Now a user authorized to use SCRAM needs that in pg_hba.conf:

HOST USER DBNAME scram

This is not portable at all. Something like that would be better:

HOST USER DBNAME sasl method=scram_sha_256

"method" needs to be a mandatory parameter, and scram_sha_256 its unique possible value to allow future extensibility.