Oracle to Postgres Conversion
From PostgreSQL wiki
Note: This page was originally carried on the OpenACS Website, but seems to have been lost over the ages and so is now reproduced here. by James Shannon, Ben Adida, and Don Baccus
What you should know before you begin
You should know SQL relatively well. Knowing the details of Oracle SQL and Postgres SQL are obviously tremendous advantages, but the hints in this document should quickly bring you up to speed on what the differences are.
If you're porting Oracle SQL to Postgres SQL for the ACS/pg, you should also be quite familiar with AOLserver Tcl, especially the AOLserver database APIs.
In this document, we're talking about:
- Oracle 8 and 8i
- Postgres 8.4, and sometimes this also works for earlier versions.
There are a handful of grammar differences in Postgres for functionality that is actually the same. ACS/pg attempts to perform these changes automatically, leaving only the major functionality differences to be ported by hand. This is done by db_sql_prep which performs a number of regular expression substitutions on a piece of SQL.
Oracle uses the keyword sysdate to denote the current date and time in server's timezone. Postgres uses 'now'::timestamp for current transaction start date and time in session's timezone, which ACS/pg has conveniently wrapped in a function named sysdate().
ACS/pg also includes a Tcl procedure named db_sysdate which should be used every time the sysdate term appears. Thus:
set now [database_to_tcl_string $db "select sysdate from dual"]
set now [database_to_tcl_string $db "select [db_sysdate] from dual"]
The Dual Table
Oracle uses the "fake" dual table for many selects, where in PostgreSQL we can write select just without from part at all. This table was created in postgres as a view to ease porting problems. This allows code to remain somewhat compatible with Oracle SQL without annoying the Postgres parser.
Oracle's sequence grammar is sequence_name.nextval.
Postgres's sequence grammar is nextval('sequence_name').
In Tcl, getting the next sequence value can be abstracted by calling [db_sequence_nextval $db sequence_name]. In case you need to include a sequence's value in a more complex SQL statement, you can use [db_sequence_nextval_sql sequence_name] which will return the appropriate grammar.
Oracle's handy decode function works as follows:
decode(expr, search, result [, search, result...] [, default])
To evaluate this expression, Oracle compares expr to each search value one by one. If expr is equal to a search, Oracle returns the corresponding result. If no match is found, Oracle returns default, or, if default is omitted, returns null.
Postgres doesn't have the same construct. It can be replicated with:
CASE WHEN expr THEN expr [...] ELSE expr END
which returns the expression corresponding to the first true predicate. For example:
CASE WHEN c1 = 1 THEN 'match' ELSE 'no match' END
Oracle has another handy function: NVL. NVL returns its first argument if it is not null, otherwise it returns its second argument.
start_date := NVL(hire_date, SYSDATE);
The above statement will return SYSDATE if hire_date is null. Postgres (and Oracle) has a function that performs the same thing in a more generalized way: coalesce(expr1, expr2, expr3,....) returns the first non-null expression that is passed to it.
Subquery in FROM
PostgreSQL requires a sub-SELECT surrounded by parentheses, and an alias must be provided for it. The alias is not mandatory for Oracle.
a query for Oracle:
SELECT * FROM (SELECT * FROM table_a)
in PostgreSQL will look like:
SELECT * FROM (SELECT * FROM table_a) as foo
Postgres doesn't have all the functionality of Oracle. ACS/pg is forced to deal with these limitations with specific work-arounds. Almost everything can be done under Postgres, but some features are awaiting new versions of the open-source database.
Outer Joins in Oracle work as follows:
select a.field1, b.field2 from a, b where a.item_id = b.item_id(+)
where the (+) indicates that, if there is no row in table b that matches the correct item_id, the match should still happen, with an empty row from table b. In this case, for example, for all rows in table a where there is no matching row in b, a row will still be returned where a.field1 is correct, but b.field2 is null.
select a.field1, b.field2 from a left outer join b on a.item_id = b.item_id;
In certain other cases where only aggregate values are pulled out of the outer-joined table, it's possible to not use a join at all. If the original query is:
select a.field1, sum(b.field2) from a, b where a.item_id = b.item_id (+) group by a.field1
then the Postgres query can look like:
select a.field1, b_sum_field2_by_item_id(a.item_id) from a
where you've defined the function:
create function b_sum_field2_by_item_id(integer) returns integer as ' DECLARE v_item_id alias for $1; BEGIN return sum(field2) from b where item_id= v_item_id; END; ' language 'plpgsql';
Postgres doesn't have connect by statements. It does, however, have WITH RECURSIVE. As WITH RECURSIVE is Turing-complete, it is simple to translate CONNECT BY statements into WITH RECURSIVE ones.
NO_DATA_FOUND and TOO_MANY_ROWS
This exceptions are disabled by default for selects in PLpgSQL. You need to add keyword STRICT after any keyword INTO in all selects, when You need to keep single row checking in stored PLpgSQL code. See documentation.
Empty strings and NULL values
In Oracle empty strings () and NULL values in string context are the same till latest versions. You can concatenate NULL result with a string and obtain this string as result. In PostgreSQL you will obtain null in this case. In Oracle You need use IS NULL operator to check whether string is empty. In PostgreSQL You will obtain FALSE for empty string and TRUE for NULL.
On conversion Oracle to PostgreSQL You will need to carefully analize all Your string handling code and separate NULLs and empty strings.
Postgres has decent CLOB support in the form of TEXT.
Binary large object support in Postgres is very poor and unsuitable for use in a 24/7 environment, because you can't dump them with pg_dump. Backing up a database that makes use of Postgres large objects requires one to knock down the RDBMS and dump the files in the database directory.
Don Baccus put together a hack that extends AOLserver's postgres driver with BLOB-like support, by uuencoding/decoding binary files before stuffing them into or extracting them from the database. The resulting objects can be consistently dumped by "pg_dump" while the RDBMS is up and running. There is no need to interrupt service while making your backup.
To get around the one-block limit on the size of a tuple imposed by Postgres, the driver segments the encoded data into 8K chunks.
Postgres large objects are scheduled for a major overhaul in summer 2000. Because of this, only the BLOB functionality used by the ACS was implemented.
To use the BLOB driver extension, you must first create a column of type "integer" with the name "lob" in the table that will store the BLOB, and a trigger on it that calls "on_lob_ref". You must use the name "lob". Here's an example:
create table my_table ( my_key integer primary key, lob integer references lobs, my_other_data some_type -- etc );
create trigger my_table_lob_trig before insert or delete or update on my_table for each row execute procedure on_lob_ref();
To put a binary file into "my_table":
set lob [database_to_tcl_string $db "select empty_lob()"] ns_db dml $db "begin" ns_db dml $db "update my_table set lob = $lob where my_key = $my_key" ns_pg blob_dml_file $db $lob $tmp_filename ns_db dml $db "end"
Note that the call to ns_pg to stuff the file into the database MUST be wrapped in a transaction, even if you're not updating any other tables at the same time. The driver will return an error if you don't.
To return a large object stored in "my_table" to the user:
set lob [database_to_tcl_string $db "select lob from my_table where my_key = $my_key"] ns_pg blob_write $db $lob
Note that you don't need to wrap the call to blob_write in a transaction, as the database isn't being modified.
The large objects are automatically deleted when no longer used. To replace the large object stored in an existing record, just allocate a new one by calling "empty_lob()" and assign the returned key to the "lob" column in your table.
A couple of extension tools are available for Oracle migration.
- Full Convert
- Database conversion between Oracle (and 30+ other database engines) and PostgreSQL. With everything on localhost, typical throughput over 100k records per second.
- Oracle to Postgres data migration and sync
- Software is able to convert 1 Million of records in 4-5 minutes. Trigger-based database sync method and simultaneous Bi-directional synchronization help you to manage your data easy and efficiently.
- ESF Database Migration Toolkit
- A toolkit migrates Oracle databae to PostgreSQL in wizard. It connects to Oracle and PostgreSQL database directly, and migrate its table structure, data, indexes, primary keys, foreign keys, comments and so on.
- The goal of the project is to implement some functions from Oracle database. Some date functions (next_day, last_day, trunc, round, ...), string functions and some modules (DBMS_ALERT, DBMS_OUTPUT, UTL_FILE, DBMS_PIPE, ...) are implemented now. Functionality was verified on Oracle 10g and module is useful for production work.
- Ora2Pg is a Perl module to export an Oracle database schema to a PostgreSQL compatible schema. It connects your Oracle database, extracts its structure, and generates an SQL script that you can load into your PostgreSQL database.
- Oracle to Postgres
- Program to migrate Oracle databases to PostgreSQL server. It does not use ODBC or any other middleware software. Table structures, data, indexes, primary keys and foreign keys are converted. Command line support allows to script, automate and schedule the conversion process.