Datadog cost surprises are more common than you'd think. I’ve written about how Datadog’s defaults can quietly spiral out of control, like when a GKE Autopilot user saw their Datadog bill increase 20x due to unexpected Prometheus scraping. The root cause? Overlapping defaults that not even
Deploying on a Friday afternoon is always a gamble. Last Friday, we pushed what seemed like a routine front-end update, until users started reporting errors. A null object slipped through, breaking part of the UI. Normally, this would trigger a bunch of Sentry alerts. But Sentry alerts are noisy, and
Spending on observability platforms has gotten out of control. It is often the number two cost behind a company’s main cloud vendor, representing up to 25% of the engineering team’s budget. I saw this firsthand when I was part of the engineering team at Coinbase, where I built
Imagine opening your Datadog bill and seeing a 20x increase. You dig deeper and discover thousands of “custom metrics” flooding into Datadog—metrics you didn’t configure, expect, or even recognize. That’s exactly what happened to a GCP user running workloads on GKE Autopilot. I’ve said this before,
Migrating from SQL Server to Postgres is a significant move for organizations looking to reduce costs, improve scalability, and future-proof their data infrastructure. But database migrations are costly and time consuming. They frequently run overtime, carry significant business risks, and are a significant opportunity cost. So if you’re going
Snowflake vs. Databricks: who’s better for your company’s AI initiatives? Competition between the two has been heating up for years, and their recent announcements at the 2024 Snowflake Summit and Databricks’ Data+AI conference reflect the intensifying shift towards enabling AI capabilities for their users. It's
In this post, we explore the limits of single-node Postgres and what optimizations can extend its usefulness as a primary application database before requiring more advanced scaling solutions. Monolithic ecommerce app on RDS PostgreSQL Let's use a real-world scenario, a monolith web application that uses PostgreSQL as the
What's missing from APMs? Sales are dropping on your ecommerce shop. Customers are getting sluggish response times when they're viewing products for sale. As a backend engineer supporting your Ruby on Rails and PostgreSQL ecommerce platform, you spring into action using your available tools. Your primary
Do you have a Rails app built on PostgreSQL and need to scale it to millions of users? In this interview, I speak with Andrew Atkinson, one of StepChange's expert consultants, who brings deep expertise in optimizing Rails applications for performance and scalability. Andrew’s expertise is backed
Having discussed why Postgres performance tuning matters, and how to write better queries to optimize performance, it’s time to look beyond the basics towards more advanced techniques in Postgres performance tuning and monitoring. I’ll share how you can use tools like pg_stat_statements for query analysis, implement
Are you thinking about migrating a Native mobile app to React Native or building one from scratch? Are you interested in hearing lessons learned from migrating Coinbase's 56 million users from Native to React Native? Nick Cherry, an Expert React Native developer and former Software Engineer at Coinbase,
In my previous article, I discussed why Postgres performance tuning matters, warning signs you should pay attention to know if your Postgres database is struggling, and 8 parameters you can try to improve its performance and see results immediately. Today, we’ll take a closer look at Postgres query optimization
News
In our work as seasoned database specialists, we often approach Postgres performance tuning with an eye towards how we can best tune a range of configuration parameters to suit a specific workload and hardware setup. But it’s not always obvious why you should start here, where you should start,
video
In this interview, I talk with Justin Watt, the founder of Switchboard, a consultancy focused on automation solutions, about how his company helps businesses automate their processes and workflows. Before founding Switchboard, Justin worked at IBM on large-scale government projects and at MetaLab with clients like Amazon and Uber. His
video
Are you thinking about building a polished AI powered app to explore a large data set? Worried about how to handle data accuracy issues? Do you want to know the pro's and con's of using pgvector with Postgres over a dedicated vector store? Are you interested
video
Do you have a Django app built on Postgres that needs to use geospatial data? Want to know how to scale it to millions of users using Postgres? In this article, I interview Ananth Chandrasekharan about the challenges Nextdoor faced while scaling its Django application, which made heavy use of
video
Do you have a Django app built on Postgres? Want to know how to scale it to 10 million users using AWS Aurora? Luke Demi, is an expert software engineer who helped scale Clubhouse’s backend infrastructure from 10 thousand users to 10 million users in 6 months. In this
By examining real-world scenarios encountered at Coinbase and Nextdoor, this post presents a technical exploration of these challenges, offering guidance to teams seeking to enhance their database operations.
TL;DR: Amazon Aurora extends PostgreSQL with better scalability and performance, enabling efficient autoscaling and Serverless options. However, it faces challenges with complex pricing and vendor lock-in. This article helps navigate those issues. Introduction As your PostgreSQL application grows in scale, whether you choose to self-host or opt for a
Current AI evaluation metrics fail to represent many aspects of real-world software engineering tasks. Introduction AI software engineers like Devin and SWE-agent are frequently compared to human software engineers. However SWE-bench, the benchmark upon which this comparison is made, only applies to Python tasks, most of which involve making single-file
StepChange is a consultancy that works with you to modernize your data infrastructure. Often this involves optimizing cloud spend. Recently there has been a lot of conversation about dbt. This article hopes to clarify what it is and how people are dealing with their price increases. What is DBT? Data
The following blog post was inspired by Astronomer's webinar on how to migrate from Oozie to Airflow by Ben Spivey and Dylan Storey. Many enterprises are looking to migrate from Apache Oozie to Airflow. For those who may not know, Apache Oozie is an established open-source workflow scheduler