Kubernetes, Hadoop, Persistent Volumes and vSAN

At VMworld 2018, one of the sessions I presented on was running Kubernetes on vSphere, and specifically using vSAN for persistent storage. In that presentation (which you can find here), I used Hadoop as a specific example, primarily because there are a number of moving parts to Hadoop. For example, there is the concept of Namenode and a Datanode. Put simply, a Namenode provides the lookup for blocks, whereas Datanodes store the actual blocks of data. Namenodes can be configured in a HA pair with a standby Namenode, but this requires a lot more configuration and resources, and introduces additional…

Fun with PKS, K8s, VCP, StatefulSets and Couchbase

After just deploying the newest version of Pivotal Container Services (PKS) and rolling out my first Kubernetes cluster (read all about it here), I wanted to try to do something a bit more interesting than just create another persistent volume claim to test out our vSphere Cloud Provider since I had done this  a number of times already. Thanks to some of the work I have been doing with our cloud native team, I was introduced to StatefulSets. That peaked my interest a little, as I had not come across them before.

See you at VMworld 2018

Yes, it’s that time of year again. VMworld 2018 in Las Vegas is taking place next month. Once again, as per previous years, I will be co-presenting on a few sessions. It will be no surprise that, once again, my sessions are focused on storage, hyper-convergence and predominantly vSAN. However, this year I will also be presenting with my CTO and VMware Fellow, Christos Karamanolis, for the very first time. Without further ado, let me go through my sessions in a bit more details, and if they look interesting to you, feel free to sign up. I also included a…

PKS Revisited – Project Hatchway / K8s vSphere Cloud Provider review

As I am going to be doing some talks around next-gen applications at this year’s VMworld event, I took the opportunity to revisit Pivotal Container Services (PKS) to take a closer look at how we can set persistent volumes on container based applications. Not only that, but I also wanted to leverage the vSphere Cloud Provider feature which is part of our Project Hatchway initiative. I’ve written about Project Hatchway a few times now, but in a nutshell this allows us to create persistent container volumes on vSphere storage, and at the same time set a storage policy on the…

PKS – Networking Setup Tips and Tricks

In my previous post, I showed how to deploy Pivotal Container Services (PKS) on a simplified flat network. In this post, I will highlight some of the issues one might encounter if you wish to deploy PKS on a more complex network topology. For example, you may have vCenter Server on a vSphere management network alongside the PKS management components (PKS  CLI client, Pivotal Ops Manager). You may then want to have another “intermediate network” for the deployment of the BOSH and PKS VMs. And then finally, you may finally have another network on which the Kubernetes (K8s) VMs (master,…

A simple Pivotal Container Service (PKS) deployment

This post will walk you through a simplified PKS (Pivotal Container Service) deployment in my lab. The reason why I say this is simplified is because all of the components will be deployed on a single flat network. PKS has a number of network dependencies. These include the bosh agents deployed on the Kubernetes (K8s) VMs being able to reach the BOSH Director, as well as the vCenter server. Let’s not get too deep into the components just yet – these will be explained over the course of the post. So rather than trying to set up routing between multiple…

Project Hatchway hitting the mainstream – persistent storage for containers

Regular readers will be aware that I “dabble” from time to time in the world of Cloud Native Apps. For me, a lot of this dabbling is trying to figure out how I can go about providing persistent storage to container based applications. Typically this in the shape of container volumes that are carved out of the underlying storage infrastructure, whether that is VMFS, NFS, vSAN or even Virtual Volumes. VMware Project Hatchway has enabled me to do this on multiple occasions. Project Hatchway was officially announced at VMworld 2017, but I’ve been working with this team since the early…