Postgres VACUUM and Xmin Horizon

I had a chance to learn about the concept called the “xmin horizon” in Postgres, so I’m leaving a learning memo. The xmin horizon is also called the oldest xmin, and it tells you “until which point (in terms of transactions) the vacuum process can clean up dead rows”. In other words, VACUUM can’t clean up dead rows after this xmin horizon. When you see a log like “1165 are dead but not yet removable”, it means that there were 1165 dead rows, but VACUUM wasn’t able to clean them up because they are after the xmin horizon.…

Keep reading

How I Got My Green Card

In May 2017, I received my Green Card (the US Permanent Residency Card). This was a long journey and it took about 2 years, so I’d like to explain how I got one and what the journey was like. Beforehand: My situation In April 2013, I got a job at an American company (Salesforce/Heroku). I started working remotely from Japan as I didn’t have a working visa at that time. I had been living in the US right before I got this job and I wanted to return to the US, so we started an L-1 visa application process.…

Keep reading

Battle With A Phantom WAL segment

While the Heroku Postgres team were trying to close “the follower hole”, we discovered “phantom WAL1 segments”. This was causing recovering standbys that were trying to replay that segment to crash or enter a retry loop. In this article, I’m going to explain what the follower hole is, what the phantom WAL segment is, and how we tackled it. The phantom WAL segment is a Postgres bug and it is patched with the Postgres release of 2017-08-10.…

Keep reading

Grailed Engineering Hangout

Early October, I had a chance to give a talk at Grailed, with a title “Healthy Database”, about Heroku Postgres and databases in general to their engineers and friends (thanks for having me, Grailed!). Grailed is a curated community marketplace for men’s clothing. They’re an NYC based company, have a very nice office and super nice people. They recently launched Heroine, women’s version of Grailed. You can find my slides here: https://speakerdeck.…

Keep reading

Write a Blog Post

Last month or so, I finally decided to write a blog post. It’s been a while since I wrote one. Basically I stopped writing something after I joined Heroku. I’ve learned many things last 5 years (well that’s the last time I wrote a blog post, seems like), and I’ve also experienced many things. Thanks to the technology and community, seems like blogging systems improved a lot. Looking at the deploy log, I’ve moved my blog from WordPress to Middleman in 2014.…

Keep reading