NoSQL to SQL - Complex App Overhaul

What Is This About?

While interviewing for a position at a company, I was asked to answer the following question: Can you please describe the hardest problem you have solved? In summary, the most difficult problem I solved occurred while I was working at eBay, where I migrated one of eBay’s large internal projects from using MongoDB to Postgres during the year 2023. However, this question really made me think about how I solved this difficult problem and allowed me to reflect on my developer journey. Due to this, I wrote a very detailed and thorough response to that question, and I wanted to share my response on my website.

What Was The Question?

Can you please describe the hardest problem you solved? Please reply with less than 4 paragraphs and let us know:

1. What you did
2. What made if difficult
3. What you learned from this experience
4. How you grew as a result

What Was My Response?

1 - What you did

Currently, in my career, the hardest problem I solved occurred well working at my current position at eBay. Currently at eBay, where I am contracted via Hitachi Vantara, I develop and maintain an internal tool called Storage Management System or STMS. STMS is a tool used by eBay’s Service & Storage Infrastructure (SSI) team to monitor and manage devices across eBay’s data centers. STMS allows the ability for an engineer to monitor the metrics of numerous arrays, switches, hosts, disk groups, clusters, and databases. As well as handle alerting for switches and arrays and easily allow an engineer to complete advanced tasks like host allocations. STMS ingests over 1.5 million metrics per minute, accounting for over dozens/hundreds of arrays, switches, hosts, disk groups, and clusters across eBay’s data centers. It serves as a critical tool for the SSI team to monitor and manage eBay’s storage infrastructure, which is vital for eBay’s core service and business model. Knowing all of this, the most difficult problem I had to solve during my time working at eBay was migrating the STMS from using the MongoDB database to using the Postgres database.

2 - What made it difficult

What made this problem difficult was that MongoDB and Postgres are fundamentally different databases. MongoDB is a document-based database (NoSQL), meaning data is stored as JSON in a collection (like documents in a filing cabinet), and Postgres is a relational database (SQL), meaning data is stored as rows in a table (like in a spreadsheet). Additionally, STMS’s entire backend was built to process and manage data as JSONs, using packages exclusively compatible with MongoDB for database operations. Also, well doing this migration, there had to be no downtime and nearly all features had to continue working due to how vital STMS is as an internal tool. This migration in its entirety had to be completed within a few months, with no downtime, and no clear plan for its execution. I also did not have too much experience with Postgres beforehand and, along with all my coworkers, did not have any experience in migrating a large legacy codebase from a NoSQL to SQL database.

3 - What you learned from this experience

The STMS code migration from MongoDB to Postgres was successfully completed with no downtime while maintaining nearly all features and functionality. This required extensive research, learning, testing, and troubleshooting. I restructured our data structures to work more effectively with relational databases and developed new packages to replace those we depended on that did not integrate with Postgres. Overall, the migration went seamlessly and without any real flaws which exceeded my expectations. Throughout this process, I learned a lot. Technically, I acquired a better understanding of Postgres, SQL, parse trees, ORMs, Sequelize, Prisma, query builders, equivalence checking, primary-standby database setup, and database-related unit testing. Fundamentally, I learned how to more effectively break down a complex problem into smaller parts and implement effective solutions for those parts. Additionally, I learned how to be a leader and lead this project to its conclusion despite facing many limitations, set backs, and unknowns.

4 - How you grew as a result

I started my career in robotics during my undergraduate but shifted towards web development when I got the job at eBay, entering a field where I had no prior experience. This transition laid the groundwork for a career defined by constant learning and adaptability. The challenge of migrating STMS from MongoDB to Postgres was a significant leap in my career, contributing immensely to my professional growth. It wasn’t just about deepening my understanding of databases or enhancing my problem-solving skills. This project pushed me into new territory, demanding meticulous planning and a commitment to no downtime. Leading the migration without previous experience in large-scale database transitions, I learned the value of clear communication, teamwork, and breaking down complex challenges into manageable parts. This experience expanded my technical skills and also boosted my confidence, gearing more towards more ambitious goals and projects. This milestone in my career not only demonstrated my growth as a developer and leader but also as a problem solver, showing that embracing the unknown and succeeding is key to personal and professional development.

What’s The Conclusion?

My responses moved me on to the second stage of the interviewing process at the company, allowing me to have a pleasant conversation with one of the developers from the team, which was pretty nice. But, mainly, this assignment allowed me to reflect on the most challenging technical problem I solved in my career during my time at eBay. It enabled me to think about my growth as a developer/engineer, develop new skills, and enhance my confidence in adapting to and solving complex problems.