Skip to main content
Last updated:|... min read
Bug iconOpen OLake issuesBug iconOpen OLake doc issue

Postgres Source

Postgres Source enables data synchronization from Postgres to your desired destination.

info

OLake UI is live (beta)! You can now use the UI to configure your Postgres source, discover streams, and sync data. Check it out at OLake UI regarding how to setup using Docker Compose and running it locally.

olake-source-postgres

Now, you can use the UI to configure your Postgres source, discover streams, and sync data.

Create a Postgres Source in OLake UI​

Follow the steps below to get started with the Postgres Source using the OLake UI (assuming the OLake UI is running locally on localhost:8000):

  1. Navigate to Sources Tab.
  2. Click on + Create Source.
  3. Select Postgres as the source type from Connector type.
  4. Fill in the required connection details in the form. For details regarding the connection details, refer to the Postgres Source Configuration section on the right side of UI.
  5. Click on Create ->
  6. OLake will test the source connection and display the results. If the connection is successful, you will see a success message. If there are any issues, OLake will provide error messages to help you troubleshoot.

This will create a Postgres source in OLake, now you can use this source in your Jobs Pipeline to sync data from Postgres to Apache Iceberg or AWS S3.

Edit Postgres Source in OLake UI​

To edit an existing Postgres source in OLake UI, follow these steps:

  1. Navigate to the Sources Tab.
  2. Locate the Postgres source you want to edit from Active Sources or Inactive Sources tabs or using the search bar.
  3. Click on the Edit button next to the source from the Actions tab (3 dots).
  4. Update the connection details as needed in the form and Click on Save Changes.
caution

Editing a source can break pipeline.

You will see a notification saying "Due to the editing, the jobs are going to get affected".

Editing this source will affect the following jobs that are associated with this source and as a result will fail immediately. Do you still want to edit the source?

olake-source-edit-2

  1. OLake will test the updated source connection once you hit confirm on the Source Editing Caution Modal. If the connection is successful, you will see a success message. If there are any issues, we will provide error messages to help you troubleshoot.

Jobs Associated with Postgres Source​

In the Source Edit page, you can see the list of jobs that are associated with this source. You can also see the status of each job, whether it is running, failed, or completed and can pause the job from the same screen as well.

olake-source-associated-job-1

Delete Postgres Source in OLake UI​

To delete an existing Postgres source in OLake UI, follow these steps:

  1. Navigate to the Sources Tab.
  2. Locate the Postgres source you want to delete from Active Sources or Inactive Sources tabs or using the search bar.
  3. Click on the Delete button next to the source from the Actions tab (3 dots).

olake-source-delete-2

  1. A confirmation dialog will appear asking you to confirm the deletion.
  2. Click on Delete to confirm the deletion.

olake-source-delete-1

This will remove the Postgres source from OLake.

note

You can also delete a source from the Source Edit page by clicking on the Delete button at the bottom of the page.

PostgreSQL to Iceberg Data Type Mapping​

When syncing data from PostgreSQL to Iceberg, OLake handles data type conversions to ensure compatibility. Below is a table that outlines how PostgreSQL data types are mapped to Iceberg data types:

PostgreSQL Data TypesIceberg Data Type
int, int2, int4, smallint, integer, serial, serial2, serial4int
bigint, bigserial, int8, serial8bigint
float, float4, real, numericfloat
double precision, float8double
booleanboolean
date, timestamp without time zone, timestamp with time zonetimestamp
box, bpchar, char, character, character varying, json, jsonb, jsonpath, name, numrange, path, text, tid, tsquery, tsrange, tstzrange, uuid, varbit, varchar, xml, inet, bit, int2vector, daterange, time with time zone, time without time zonestring

Changelog​

Expand to review

Need Assistance?

If you have any questions or uncertainties about setting up OLake, contributing to the project, or troubleshooting any issues, we’re here to help. You can:

  • Email Support: Reach out to our team at hello@olake.io for prompt assistance.
  • Join our Slack Community: where we discuss future roadmaps, discuss bugs, help folks to debug issues they are facing and more.
  • Schedule a Call: If you prefer a one-on-one conversation, schedule a call with our CTO and team.

Your success with OLake is our priority. Don’t hesitate to contact us if you need any help or further clarification!

Join our growing community
GitHub
Slack
Twitter
LinkedIn
YouTube