Josh Peng

Full-Stack Data Scientist

github linkedin email
Heroku Postgres <> Fivetran
May 12, 2019
One minute read

Turns out Heroku Postgres and Fivetran don’t play all too well with each other.

The main issue is that Heroku Postgres doesn’t give you superuser access which means you can’t create replication slots. Without replication slots, Fivetran has an incredibly hard time trying to replicate data into your warehouse provider because it has to fallback to an alternate method reliant on logic built around XMIN.

XMIN comes with several disadvantages: 1. Inability to detect deleted rows. All rows have _fivetran_deleted = False which means, over time, you will have incorrect row counts in your tables. 2. Incredibly poor performance. It takes magnitudes longer to replicate data when compared against a replication slot and is incredibly unstable. I’ve been on Fivetran support for over a week trying to stabilize a job from Heroku Postgres to an AWS hosted Postgres solution.

If possible, I would recommend staying away from Heroku Postgres. Amazon RDS for Postgres is a superior alternative.

Tags: #postgres

Back to posts

comments powered by Disqus