Databases on Kubernetes offer flexibility and scalability, but understanding the fundamentals is key to successful implementation. In this article:”How to Provision a MongoDB Cluster in Kubernetes”, we’ll delve into the insights from Peter’s presentation within a recent webinar.

He’ll guide us through the advantages and challenges of Kubernetes database deployment, comparisons between cloud solutions, and the best open-source options. 

Remember two years ago, at conferences, every conversation about databases on Kubernetes started the same way: “Really? Who would do that?” The idea seemed crazy – Kubernetes, known for its fleeting containers, wasn’t built for the world of persistent databases, right? It felt like a flashback to the early days of virtual machines, when everyone questioned their suitability for databases.

Fast forward to today. The scene at KubeCon is a revelation. People are no longer asking “if” but “how.” They’ve experimented, they have questions, and they’re eager to unlock the power of Kubernetes for their databases. This shift in perspective reflects a key trend: Kubernetes is maturing.

The ecosystem is booming, the community is thriving thanks to the open-source nature, and new use cases are emerging – like leveraging AI and machine learning with specialized hardware like GPUs. It’s an exciting time, and Kubernetes is proving its adaptability beyond its initial design.

From panel discussions to audience questions, it seems like the buzz around databases on Kubernetes is undeniable. So, what exactly is fueling this shift? Well, let’s unpack a few key factors:

• Busting the “ephemeral” myth: StatefulSets and persistent volumes have been game-changers. They’ve addressed a major concern about the transient nature of containers, paving the way for reliable database implementations.
• Operators: the rise of automation: The market is flourishing with mature operators for almost any database technology you can imagine. These operators streamline deployment and management within Kubernetes, making the process far less daunting.
• Kubernetes superpowers: Kubernetes’ native features like high-availability and disaster recovery are a perfect fit for databases that demand resilience and uptime. Plus, who doesn’t love the scalability and flexibility that Kubernetes offers right out of the box?
• The CSI revolution (behind the scenes): Here’s something more technical that I found fascinating: the decoupling of CSI drivers from the core of Kubernetes. This may not sound like much, but it’s spurred incredible growth in available CSI drivers, ultimately improving the overall experience for database deployments.
  

Of course, it’s not all rainbows and unicorns. Challenges remain, but these advancements show undeniable progress.

Let’s be real – no technology is perfect, and databases on Kubernetes are no exception. This is where the classic IT phrase “it depends” rings true. But there are a few recurring challenges worth highlighting:

• The complexity monster: Change can be scary, especially when you’re adding Kubernetes to the already complex world of databases. The fear is understandable; it’s a whole new beast to master on top of existing knowledge. This hesitancy is a major obstacle to adoption.
• Lessons from failed experiments: Some organizations have dipped their toes in the water, only to get burned. These early failures create lingering skepticism – it’s hard to justify “trying again” with less-than-stellar results. Unfortunately, this often stems from a lack of internal Kubernetes know-how. It’s a different world, and success requires new skills.
• Massive data: a special case: While technologies are emerging to tackle large datasets on Kubernetes, it remains a valid concern. When you’re talking databases reaching into hundreds of terabytes (or whatever massive scale blows your mind), finding the right fit is essential.

It’s important to acknowledge these challenges. Understanding the pitfalls helps us navigate them more effectively in the future!

Public vs private DBaaS showdown: when to DIY, when to outsource

Databases on Kubernetes offer automation and the allure of a self-service model – it practically screams “database-as-a-service” (DBaaS), doesn’t it? But when you talk to those in the trenches, there’s a clear pattern emerging: the DBaaS decision often comes down to a few key factors driving people towards these ready-made solutions, whether it’s hyperscalers like RDS Aurora or some other flavor of DBaaS.

• Need for speed: Quick and easy database spin-up is a major draw. If your internal ops teams are a bottleneck, with dev teams constantly complaining about delays, DBaaS can look mighty tempting.
• The hyperscaler appeal: The big names in cloud (think RDS Aurora) offer robust solutions that are hard to resist… unless you have very specific requirements or you’re operating at a scale where the costs of a hyperscaler outweigh the benefits of managing your own infrastructure.

Hyperscalers are shiny and convenient, but beneath the surface, there are some real challenges to consider:

• Security and privacy: a double-edged sword: Sure, hyperscalers generally have tight security (and plenty of brilliant people behind it), but breaches do happen. Some organizations have such strict rules that even the slightest risk is enough to say “no thanks.”
• Customization limits: you’re stuck in their sandbox: It’s true – most hyperscaler DBaaS offerings give you only basic knobs and dials. If your solution demands deeper customization that the underlying technology allows, you’re essentially getting a black box where your options are extremely limited.
• The noisy neighbor effect: Performance variability is a real issue with multi-tenant environments. Even with all the tech wizardry hyperscalers use, sometimes things go haywire unexpectedly. I’ve personally seen whole products go down because another company in the same datacenter got throttled, killing traffic for everyone!
• Compliance nightmares: If your organization faces strict compliance hurdles…well, hyperscalers might not be the answer, or you’ll end up paying a hefty price to make it work.
• Vendor lock-in: the hidden cost: This fear is growing. It’s great at first: you move everything, and it runs smoothly for a while. But down the line, you realize you’re stuck unless you want to embark on the epic quest of migrating away from that vendor. More and more companies are waking up to this, valuing the freedom to switch platforms if needed.

Public DBaaS might seem tempting, but what if you crave more control? Enter the world of private DBaaS – building your own service on Kubernetes. It sounds daunting, but with great operators and internal Kubernetes expertise, it’s achievable. Many companies have successfully built and delivered exceptional private DBaaS solutions.

Here’s why some organizations choose this path:

• Unleashing flexibility: The allure of private DBaaS lies in its flexibility. You’re not tied to someone else’s limitations or unchangeable fields. You get to call the shots.
• Hiding the complexity, not the power: While hyperscalers like RDS and Aurora are popular, they can be tricky to deploy initially, especially without a strong technical background. Private DBaaS lets you offer a user-friendly experience while maintaining full control behind the scenes.
• Seamless integration: Integration with your existing environment – CI/CD, internal cloud platforms, etc. – becomes a breeze with your own private DBaaS. It becomes a natural extension of your workflow.
• Compliance, security, and peace of mind: Private DBaaS gives you complete control over compliance, security policies, and eliminates the “noisy neighbor” risk that comes with multi-tenant environments. It’s your data, your rules.

Building your own private DBaaS requires effort, but the rewards in terms of flexibility, security, and control can be significant for some organizations.

Okay, let’s recap why a private DBaaS is worth considering:

• Security superhero: It’s your infrastructure, your rules. You become the master of your data’s security and compliance.
• Customization cravings satisfied: Tweak versions, updates, extensions, whatever! You finally escape the confines of those limited hyperscaler settings.
• The joy of predictability: No more random performance hiccups from noisy neighbors. Your databases get the resources they deserve.
• Compliance compliance everywhere: Your data sovereignty is absolute.
• Scaling up, costing down: Especially at a large scale, private DBaaS can become more budget-friendly than those hyperscalers.

As we’ve explored, deploying databases on Kubernetes presents both exciting opportunities and unique challenges. The technology is maturing rapidly, spurred by tools like StatefulSets, robust database operators, and Kubernetes’ built-in advantages. While there’s always a learning curve, the benefits are clear: enhanced automation, flexibility, and the possibility of building your own private DBaaS for greater control.

It is important to note that this is merely a preliminary examination of the extensive universe of databases on Kubernetes.. Stay tuned for the second part of our article series: ”How to Provision a MongoDB Cluster in Kubernetes”,  where Diogo will take you on a hands-on deep dive. You’ll see a demo of MongoDB deployment on Kubernetes, explore backup and recovery strategies, and gain even more practical insights.

We extend our heartfelt gratitude to the speakers Peter Szczepaniak and Diogo Recharte for their insightful contributions to this discussion. Watch the full webinar here. Join us, Document Database Community on Slack, and share your comments below.

Related posts