Posts Tagged - Kubernetes

SuperGloo to the Rescue! Making it easier to write extensions for Service Meshes.

SuperGloo

Overview

SuperGloo is a new open source project that helps with managing service meshes at scale. SuperGloo provides an opinionated abstraction layer that simplifies the installation, management, and operations of one or more service meshes like Istio, AWS App Mesh, Linkerd and HashiCorp Consul. Supports on-site, in the cloud or any combination you need.

SuperGloo Architecture

There are a growing number of articles on why SuperGloo, like Christian Posta’s “Solo.io Streamlines Service Mesh and Serverless Adoption for Enterprises in Google Cloud”. This article is going to focus on how SuperGloo can help software packages like Weaveworks Flagger work on multiple Services Meshes like Istio and AWS App Mesh that both support traffic shifting.

Flagger is a cool open source project that automates the promotion of Canary deployments of your Kubernetes services. You associate a Flagger Canary Kubernetes custom resource (CRD) with your deployment and Flagger follows your defined rules for helping roll out a new version. It detects when a new version of your service has deployed, instantiating your new version in parallel to your existing version, slowly shifting request traffic between the two versions, and using your defined Prometheus metric health checks to determine if Flagger should continue moving more traffic to the new version or roll back to the old version. Since a Canary CRD is a YAML file, this provides you a declarative way to ensure that all of your service upgrades follow your prescribed sophisticated rollout strategy and complements GitOps pipelines used in Weave Flux and JenkinsX.

More information on what Canary deployments and Traffic Shifting are is in the following posts. Gloo uses the same underlying data plane technology - Envoy - as Istio to provide traffic shifting capabilities used by Flagger and Knative. Gloo is an API/Function gateway and not a full Service Mesh, so Gloo can be used in use cases that do not require all of the power, and weight, of full service mesh implementations.

This article quickly runs through setting up the Flagger podinfo example extension on SuperGloo with Istio so you all can see what’s involved and try yourselves if you like.

Install Kubernetes and Helm

The first step on our journey is to get a basic local Kubernetes cluster running. My friend Christian Hernandez clued me in on kind (Kubernetes IN Docker) from the Kubernetes sig-testing team. It’s a fast, lightweight way to spin up/down a local cluster assuming you have a locally running copy of Docker, e.g., Docker Desktop. This example works equally well in minikube if you prefer. The following code does the basics you need for most Kubernetes clusters.

  • Creates a kind cluster with one control plane node and one worker node
  • Configures the KUBECONFIG as kind creates a separate kubeconfig file for each kind cluster
  • Installs Helm and Tiller with a service account for Tiller

Install SuperGloo and SuperGloo installs and configures Istio

Here’s where the magic happens so let’s spend a little time teasing out all the things that are happening due to these few lines of code.

The first command supergloo init installs SuperGloo into your Kubernetes cluster that is equivalent to using Helm to install SuperGloo.

The second command kubectl --namespace supergloo-system rollout status deployment/supergloo --watch=true is a hack to wait till the SuperGloo deployment is fully deployed and running. It’s similar to using the --wait option on a Helm install.

The supergloo install istio ... command declares a custom resource and the SuperGloo controller installs and configures Istio as declared. In this case, we are installing Istio version 1.0.6 with Istio’s Prometheus installation and with Istio deploying sidecars in all pods within namespaces with the label istio-injection=enabled, i.e., Istio’s default behavior for auto-injecting sidecars. This imperative supergloo install istio command creates the following manifest that you could kubectl apply if you prefer. Refer to the full Install specification for more details.

Install Flagger

The following install Flagger and its dependent parts. The following is a quick summary of installing Flagger. More details at Flagger Doc Site.

  1. Add a reference to Flagger helm repo
  2. Wait for Tiller to be fully running. Only an issue for quick scripts that create Kubernetes clusters from scratch
  3. Create a cluster role binding that allows Flagger to modify SuperGloo/Istio resources
  4. Install core Flagger referencing Istio’s provided Prometheus and telling Flagger that SuperGloo is the mesh controller
  5. Install Flagger’s Grafana dashboards which are not used as part of this demo
  6. Install Flagger’s LoadTester which can help generate test traffic during a Canary deployment if there is not enough user traffic

Install Flagger example application

The example application podinfo is a simple golang web application. It is instrumented with Prometheus so we can tell if it’s performing well (or not) that helps with our Canary deployment to validate that the new version is handling incoming traffic. The example application also has hooks to allow you to generate faults if you want to explicitly fail a deployment to see how the Flagger Canary handles that situation. Full details on the options around the Flagger example application are here. The following is the summary of installation steps.

  1. Install a test namespace, the example Kubernetes Deployment manifest and an (optional) horizontal pod autoscaler
  2. Deploy the Canary policy for the example application. More details on that in a moment
  3. Wait for the Canary controller to report that it’s fully ready, which means Istio and Flagger are fully deployed and running

The Canary manifest has a target reference that associates it with the podinfo deployment. The Canary analysis says that for every interval (1 minute) Flagger increment by stepWeight (10%) more request traffic to the new version up to maxWeight (50%) as long as the metrics stay within the defined healthy ranges. If more than threshold (5) health checks fail, then rollback to 100% of traffic to the old version and delete the new version deployment. There is also an optional section to allow the Flagger loadtester to generate additional traffic to help validate the new Canary version, i.e., hard to know if the new version works if it has not handled any requests.

Deploy a new image version and watch the Canary deployment

First, we check on the currently deployed image version and print that out to help us verify if the test updates the service like we expect; should be quay.io/stefanprodan/podinfo:1.4.0. Then to help make the changes more visible, we trigger a background process to update the image version to quay.io/stefanprodan/podinfo:1.4.1 after a five second delay. We then loop and print out the change events for podinfo to see the traffic weight changes until the Canary reports Success. You’d need to change this loop if you want to try introducing errors to see the Canary rollback. Lastly, we’ll print out the image version which should be quay.io/stefanprodan/podinfo:1.4.1 is everything succeeded.

Cleanup Kubernetes

The final step is to clean up the Kubernetes cluster, which in our case means to delete the kind cluster by running kind delete cluster and unsetting the KUBECONFIG environment variable.

Everything

Here’s an Asciinema screen recording of the whole example script running, and afterward you can see the whole script if you wanted to try yourself. The Asciinema recorder speeds up any long running commands, that is, if a command takes more than two seconds to execute the playback delays up to two seconds. This speedup reduces the run time from 15+ minutes to around two minutes.

Summary

Hopefully, this example gave you a taste of how SuperGloo supports a Canary deployment engine like Flagger. Before SuperGloo you’d either need to learn how to install all of Istio yourself or be constrained to using a managed Istio or App Mesh installation from GKE or AWS respectively. Those are good managed offerings, but they do limit your choices to the versions and configurations they currently support.

SuperGloo provides a great abstraction and management layer to help extensions leverage one or more Service Meshes without needing to get deep into the weeds of the huge API surface area of any one of the meshes like Istio or App Mesh. SuperGloo makes it easy for applications to use just what they need of the underlying meshes. That helps with adoption on Service Meshes based on the feedback I’ve heard, that is, many are currently experimenting with Istio or App Mesh or Linkerd for just on capability, typically traffic shifting or mutual TLS, and they’re finding it difficult to manage and configure the whole mesh even though they aren’t using those other capabilities. SuperGloo to the rescue to help make it easier to use just the parts of Service Meshes that add value today, and allow you to add more as you need it including mixing and matching multiple service meshes easily to get the biggest return on your investment.

I strongly encourage you to learn more yourself as its fun to learn new technology, especially tech that helps you solve complex challenges and accelerates your ability to deploy larger and more sophisticated systems.

Read More

Kubernetes Ingress Past, Present, and Future

Photo by Luke Porter.

Overview

This post was inspired by listening to the February 19, 2019, Kubernetes Podcast, “Ingress, with Tim Hockin.” The Kubernetes Podcast is turning out to be a very well done podcast overall, and well worth the listen. In the Ingress episode, the podcasters interview Tim Hockin who’s one of the original Kubernetes co-founders, a team lead on the Kubernetes predecessor Borg/Omega, and is still very active within the Kubernetes community such as chairing the Kubernetes Network Special Interest Group that currently own the Ingress resource specification. Tim talks in the podcast about the history of the Kubernetes Ingress, current developments around Ingress, and proposed futures. This talk inspired me to reflect on both Ingress Controllers (realizes the implementation of Ingress manifest) and Ingress the concept (allow client outside the Kubernetes cluster to access services running in the Kubernetes cluster).

So what’s a Kubernetes Ingress?

To paraphrase from the Kubernetes Ingress documentation, Ingress is an L7 network service that exposes HTTP(S) routes from outside to inside a Kubernetes cluster. A Kubernetes cluster may have one or more Ingress Controllers running, and each controller manages service reachability, load balancing, TLS/SSL termination, and other services for that controller’s associated routes.

Gloo as Ingress

Each Ingress manifest includes an annotation that indicates which Ingress controller should manage that Ingress resource. For example, to have Solo.io Gloo manage a specific Ingress resource, you would specify the following. Note the included annotation kubernetes.io/ingress.class: gloo.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: gloo
  labels:
    chart: jsonplaceholder-v0.1.0
  name: jsonplaceholder-jsonplaceholder
  namespace: default
spec:
  rules:
  - host: gloo.example.com
    http:
      paths:
      - path: /.*
        backend:
          serviceName: jsonplaceholder-jsonplaceholder
          servicePort: 8080

Ingress Challenges

Ingress has existed as a beta extension since Kubernetes 1.1, and it’s proven to be a lowest common denominator API. For example, the NGNIX community Ingress Controller is used by many in production, but that NGNIX Ingress controller requires the use of many NGNIX specific Ingress Annotations for all but the simplest use cases. The current Kubernetes Ingress resource specification has many limitations like that all referenced services and secrets MUST be in the same namespace as the Ingress, i.e., no cross namespace referencing. And there have been long debates about how exactly to interpret the path attribute; is it a regular expression like the documentation implies OR is it a path prefix like some controllers like NGNIX implement. These challenges have made it, in practice, difficult to have an Ingress manifest that is portable across implementations. The current Ingress manifest has also proven difficult to round trip sync with Custom Resources (CRD) which is unfortunate as CRDs are proving to be a beneficial way to extend Kubernetes.

What’s Next for Ingress?

In the podcast, Tim Hockin says given how many are using the current beta Ingress spec in production, there is a push to move the existing Ingress spec to GA status, and then start work on a next-generation specification, either an Ingress v2 or breaking up Ingress across multiple CRDs. Tim mentions how the Kubernetes community is looking at several Envoy based Ingress implementations for inspiration for the next generation of Ingress. For example, Heptio Contour has created a very interesting, and implementation neutral CRD called Ingress Route. An Ingress Route looks to address the governance challenges with Ingress, for example, if a company wants to expose a /eng route path there are many challenges with the current Ingress model as you can have conflicting Ingress manifests for the route /eng. Ingress Route provides a way to create governance and delegation such as cluster admins can define a virtual host /eng and delegate implementation explicitly to the eng namespace, and this prevents others from overriding that route path.

The Istio community, also based on Envoy like Heptio Contour, are also defining Ingress CRDs.

It will be fascinating to see how Ingress evolves in the not too distant future.

Related reading: API Gateways are going through an identity crisis.

Demo Time

I find it helpful to see working code to help make concepts more real, so let’s run through a few examples of Ingress and beyond.

For this example, I’m going to use a Kubernetes service created from https://jsonplaceholder.typicode.com/, which provides a quick set of REST APIs that provide different JSON output that can be helpful for testing. It’s based on a Node.js json-server - it’s very cool and worth looking at independently. I forked the original GitHub jsonplaceholder repository, ran draft create on the project, and made a couple of tweaks to the generated Helm chart. Draft is a super fast and easy way to bootstrap existing code into Kubernetes. I’m running all of this example locally using minikube.

The jsonplaceholder service comes with six common resources each of which returns several JSON objects. For this example, we’ll be getting the first user resource at /users/1.

  • /posts 100 posts
  • /comments 500 comments
  • /albums 100 albums
  • /photos 5000 photos
  • /todos 200 todos
  • /users 10 users

Following is a script to try this example yourself, and there’s also an asciinema playback so you can see what it looks like running on my machine. We’ll unpack what’s happening following the playback.

# Install tooling
brew update
brew cask install minikube
brew install kubernetes-cli \
  kubernetes-helm \
  azure/draft/draft \
  glooctl

# Create and set up local Kubernetes Cluster
minikube start
helm init
draft init
glooctl install ingress

# Draft runs better locally if you configure
# against minikube docker daemon
eval $(minikube docker-env)

# Get and run the example
git clone https://github.com/scranton/jsonplaceholder.git
cd jsonplaceholder
draft up

# Validate all is running
kubectl get all --namespace default
kubectl get all --namespace gloo-system
kubectl get ingress --namespace default
curl --header "Host: gloo.example.com" \
  $(glooctl proxy url --name ingress-proxy)/users/1

What Happened?

We installed local tooling (you can check respective websites for full install details)

We then started up a local Kubernetes cluster (minikube) and initialized Helm and Draft. We also installed Gloo ingress into our local cluster.

We then git clone our example and used draft up to build and deploy it to our cluster. Let’s spend a minute on what happened in this step. I originally forked the jsonplaceholder GitHub repository and ran draft create against its code. Draft autodetects the source code language, in this case, Node.js, and creates both a Dockerfile that builds our example application into an image container and creates a default Helm chart. I then made a few minor tweaks to the Helm chart to enable its Ingress. Let’s look at that Ingress manifest. The main changes are the addition of the ingress.class: gloo annotation to mark this Ingress for Gloo’s Ingress Controller. And the host is set to gloo.example.com, which is why our curl statement set curl --header "Host: gloo.example.com".


apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: 
  labels:
    chart: "-"
  annotations:
    kubernetes.io/ingress.class: 
spec:
  rules:
  - host: 
    http:
      paths:
      - path: /.*
        backend:
          serviceName: 
          servicePort:
charts/template/ingress.yaml

For more examples of using Gloo as an basic Ingress controller you can check out Kubernetes Ingress Control using Gloo.

You may have also noticed the call to $(glooctl proxy url --name ingress-proxy) in the curl command. This is needed when you’re running in a local environment like minikube and you need to get the host IP and port of the Gloo proxy server. When Gloo is deployed to a Cloud Provider like Google or AWS, then those environments would associate a static IP and allow port 80 (or port 443 for HTTPS) to be used, and that static IP would be registered with a DNS server, i.e., when Gloo is deployed to a cloud-managed Kubernetes you could do curl http://gloo.example.com/users/1.

Ingress Example Challenges

Let’s say we wanted to remap the exiting /users/1 to /people/1 as users are people too. That becomes tricky with Ingress manifests as we can set up a second rule for /people, but we need to rewrite that path to /users before sending to our service as it doesn’t know how to handle requests for /people. If you were using the NGNIX ingress, you could add another annotation nginx.ingress.kubernetes.io/rewrite-target: /, but now we’re adding implementation specific annotations, that is, the nginx annotation won’t be recognized by other Ingress Controllers. And annotations are a flat name space so adding lots of annotations can get quite messy, which is part of why Custom Resources (CRD) was created. Let’s see what the original route, and our path re-writing route, would look like in a CRD based Ingress: Gloo.

Gloo Virtual Services

Gloo uses a concept called Virtual Service that is derived from similar ideas in Istio and Envoy and is conceptually equivalent to an Ingress resource. Easiest to show you the equivalent of the example Ingress we’ve created so far in a Gloo Virtual Service.

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: default
  namespace: gloo-system
spec:
  virtualHost:
    domains:
    - gloo.example.com
    routes:
    - matcher:
        prefix: /
      routeAction:
        single:
          upstream:
            name: default-jsonplaceholder-jsonplaceholder-8080
            namespace: gloo-system

You’ll notice that it looks very similar to the Ingress we had previously created with a few subtle changes. The path specifier is prefix: / which is generally what people intend, i.e., if the beginning of the request message path matches the route path specifier than apply the route actions. If we wanted to exactly match the previous Ingress, we could use regex: /.* instead. Virtual Services allow you to specify paths by: prefix, exact, and regular expression. You can also see that instead of backend: with serviceName and servicePort, a Virtual Service has a routeAction that delegates to a single upstream. Gloo upstreams are auto-discovered and can refer to Kubernetes Services AND REST/gRPC function, cloud functions like AWS Lambda and Google Functions, and other external to Kubernetes services.

More details on Gloo at:

Let’s go back to our example, and update our Virtual Service to do the path rewrite we wanted, i.e., /people => /users

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: default
  namespace: gloo-system
spec:
  virtualHost:
    domains:
    - gloo.example.com
    routes:
    - matcher:
        prefix: /people
      routeAction:
        single:
          upstream:
            name: default-jsonplaceholder-jsonplaceholder-8080
            namespace: gloo-system
      routePlugins:
        prefixRewrite:
          prefixRewrite: /users
    - matcher:
        prefix: /
      routeAction:
        single:
          upstream:
            name: default-jsonplaceholder-jsonplaceholder-8080
            namespace: gloo-system

We’ve added a second route matcher, just like adding a second route path in an Ingress, and specified prefix: /people. This will match all requests that start with /people, and all other calls to the gloo.example.com domain will be handled by the other route matcher. We also added a routePlugins section that will rewrite the request path to /users such that our service will now correctly handle our request. Route Plugins allow you to perform many operations on both the request to the upstream service and the response back from the upstream service. Best shown with an example, so for our new /people route let’s also transform the response to both add a new header x-test-phone with a value from the response body, and let’s transform the response body to return a couple of fields: name, and the address/street and address/city.

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  creationTimestamp: "2019-04-08T21:43:45Z"
  generation: 1
  name: default
  namespace: gloo-system
  resourceVersion: "772"
  selfLink: /apis/gateway.solo.io/v1/namespaces/gloo-system/virtualservices/default
  uid: 6267ee31-5a47-11e9-bc30-867df7be8a8a
spec:
  virtualHost:
    domains:
    - gloo.example.com
    routes:
    - matcher:
        prefix: /people
      routeAction:
        single:
          upstream:
            name: default-jsonplaceholder-jsonplaceholder-8080
            namespace: gloo-system
      routePlugins:
        prefixRewrite:
          prefixRewrite: /users
        transformations:
          responseTransformation:
            transformation_template:
              body:
                text: '{ "name": "", "address":
                  { "street": "",
                    "city": "" } }'
              headers:
                x-test-phone:
                  text: ''
    - matcher:
        prefix: /
      routeAction:
        single:
          upstream:
            name: default-jsonplaceholder-jsonplaceholder-8080
            namespace: gloo-system

Let’s see what that looks like. My example GitHub repository already included the full Gloo Virtual Service we just examined. We need to configure Gloo for gateway which means adding another proxy to handle Virtual Services in addition to Ingress resources. We’ll use draft up to ensure our example is fully deployed including the full Virtual Service, and then we’ll call both /users/1 and /people/1 to see the differences.

# Install Gloo and update example
glooctl install gateway
draft up

# Call service
curl --verbose --header "Host: gloo.example.com" \
  $(glooctl proxy url --name gateway-proxy)/users/1

curl --verbose --header "Host: gloo.example.com" \
  $(glooctl proxy url --name gateway-proxy)/people/1

Mind Blown

Ok, well not that mind-blowing if you’ve used other L7 networking products or done other integration work, but still pretty cool relative to standard Ingress objects. Gloo is using Inja Templates to process the JSON response body. More details in the Gloo documentation.

Summary

In this article, we touched on some of the history and difficulties with the existing Kubernetes Ingress resources. Ingress resources continue to play a role within Kubernetes deployments despite the many challenges that annotation-based extensions have. Kubernetes Custom Resources (CRDs) was created to address some of those extension challenges and can provide a cleaner way to extend Kubernetes as you saw in the Gloo Ingress and Gateway examples. I’m a big believer in the potential of Envoy based solutions as are others in the Istio and Contour communities, and it will be exciting to see how the Kubernetes community decides to evolve Ingress after they finally move the existing resource spec to GA status.

Read More

Automating your Services with Knative and Solo.io Gloo

Knative is talked about a great deal, especially around how its capabilities can help provide more standard building blocks on top of Kubernetes for building microservices and serverless like services, e.g., scale to zero, and scale on demand. Knative high level has three capability areas: building, serving, and eventing. This post will provide some examples around Knative Build and Knative Serving with Solo.io Gloo.

Knative Serving initially included all of Istio only to use a small fraction of its capabilities around being a Kubernetes cluster ingress. Recently the Knative team added Solo.io Gloo as an alternative to Istio. More details are available in Gloo, Knative and the future of Serverless and Gloo, by Solo.io, is the first alternative to Istio on Knative.

This post shows a quick example of Knative Building, Knative Serving, and Gloo integration.

All of the Kubernetes Manifests are located in the following GitHub repository https://github.com/scranton/helloworld-knative. I encourage you to fork that repository to help you try these examples yourself.

Setup

These instructions assume you are running on a clean, recent minikube install locally, and that you also have kubectl available locally.

Install Gloo

On Mac or Linux, the quickest option is to use Homebrew. Full Gloo install instructions at Gloo documentation.

brew install glooctl

Then assuming you’ve got a running minikube, and kubectl set up against that minikube instance, i.e., kubectl config current-context returns minikube, run the following to install Gloo with Knative Serving.

glooctl install knative

Deploy existing example image

I’ve already built this example, and have hosted the image publicly in my Docker Hub repository. To use Knative to serve up this existing image, you need to do the following command.

kubectl apply --filename service.yaml

Verify the domain URL for the service. It should be helloworld-go.default.example.com.

kubectl get kservice helloworld-go \
  --namespace default \
  --output=custom-columns=NAME:.metadata.name,DOMAIN:.status.domain

And call the service. Note: the curl --connect-to option is only required when calling locally against minikube as that option will add the correct host and sni headers to the request, and send the request to the host and port pair returned from glooctl proxy address.

curl --connect-to helloworld-go.default.example.com:80:$(glooctl proxy address --name clusteringress-proxy) http://helloworld-go.default.example.com

To clean up, delete the resources.

kubectl delete --filename service.yaml

Build locally, and deploy using Knative Serving

Run docker build with your Docker Hub username.

docker build -t ${DOCKER_USERNAME}/helloworld-go .
docker push ${DOCKER_USERNAME}/helloworld-go

Deploy the service. Again, make sure you updated username in service.yaml file, i.e., replace image reference docker.io/scottcranton/helloworld-go with your Docker Hub username.

kubectl apply --filename service.yaml

Verify domain URL for service. Should be helloworld-go.default.example.com.

kubectl get kservice helloworld-go \
  --namespace default \
  --output=custom-columns=NAME:.metadata.name,DOMAIN:.status.domain

And test your service.

curl --connect-to helloworld-go.default.example.com:80:$(glooctl proxy address --name clusteringress-proxy) http://helloworld-go.default.example.com

To clean up, delete the resources.

kubectl delete --filename service.yaml

Build using Knative Build, and deploy using Knative Serving

To install Knative Build, do the following. I’m using the kaniko build template, so you’ll also need to install that as well.

kubectl apply \
  --filename https://github.com/knative/build/releases/download/v0.4.0/build.yaml

kubectl apply \
  --filename https://raw.githubusercontent.com/knative/build-templates/master/kaniko/kaniko.yaml

To verify the Knative Build install, do the following.

kubectl get pods --namespace knative-build

I’d encourage forking my example GitHub repository https://github.com/scranton/helloworld-knative, so you can push code changes and see them in your environment.

Create a Kubernetes secret for your Docker Hub account that will allow Knative build to push your image. You also need to annotate the secret to indicate it’s for Docker. More details in Guiding credential selection.

kubectl create secret generic basic-user-pass \
  --type="kubernetes.io/basic-auth" \
  --from-literal=username=${DOCKER_USERNAME} \
  --from-literal=password=${DOCKER_PASSWORD}

kubectl annotate secret basic-user-pass \
  build.knative.dev/docker-0=https://index.docker.io/v1/

It should result in a secret like the following.

kubectl describe secret basic-user-pass

Name:         basic-user-pass
Namespace:    default
Labels:       <none>
Annotations:  build.knative.dev/docker-0: https://index.docker.io/v1/

Type:  kubernetes.io/basic-auth

Data
====
username:  12 bytes
password:  24 bytes

Verify that serviceaccount.yaml references your secret.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: build-bot
secrets:
  - name: basic-user-pass

Update service-build.yaml with your GitHub and Docker usernames. This manifest will use Knative Build to create an image using the kaniko-build build template and deploy the service using Knative Serving with Gloo.

apiVersion: serving.knative.dev/v1alpha1
kind: Service
metadata:
  name: helloworld-go
  namespace: default
spec:
  runLatest:
    configuration:
      build:
        apiVersion: build.knative.dev/v1alpha1
        kind: Build
        metadata:
          name: kaniko-build
        spec:
          serviceAccountName: build-bot
          source:
            git:
              url: https://github.com/{ GitHub username }/helloworld-knative
              revision: master
          template:
            name: kaniko
            arguments:
              - name: IMAGE
                value: docker.io/{ Docker Hub username }/helloworld-go
          timeout: 10m
      revisionTemplate:
        spec:
          container:
            image: docker.io/{ Docker Hub username }/helloworld-go
            imagePullPolicy: Always
            env:
              - name: TARGET
                value: "Go Sample v1"

To deploy, apply the manifests.

kubectl apply \
  --filename serviceaccount.yaml \
  --filename service-build.yaml

Then you can watch the build and deployment happening.

kubectl get pods --watch

Once you see all the helloworld-go-0000x-deployment-.... pods are ready, then you can Ctrl+C to escape the watch, and then test your deployment.

Verify the domain URL for service. Should be helloworld-go.default.example.com.

kubectl get kservice helloworld-go \
  --namespace default \
  --output=custom-columns=NAME:.metadata.name,DOMAIN:.status.domain

And test your service.

curl --connect-to helloworld-go.default.example.com:80:$(glooctl proxy address --name clusteringress-proxy) http://helloworld-go.default.example.com

Cleanup

kubectl delete \
  --filename serviceaccount.yaml \
  --filename service-build.yaml

kubectl delete secret basic-user-pass

Summary

Hopefully, this post gave you a taste for how Gloo and Knative can work together to provide you with a way to build and deploy your services on demand into Kubernetes.

See Also

Read More

Kubernetes Ingress Control using Gloo

Kubernetes is excellent and makes it easier to create and manage highly distributed applications. A challenge then is how do you share your great Kubernetes hosted applications with the rest of the world. Many lean towards Kubernetes Ingress objects and this article will show you how to use the open source Solo.io Gloo to fill this need.

Gloo as Ingress

Gloo is a function gateway that gives users many benefits including sophisticated function level routing, and extensive service discovery with the introspection of OpenAPI (Swagger) definitions, gRPC reflection, Lambda discovery and more. Gloo can act as an Ingress Controller, that is, by routing Kubernetes external traffic to Kubernetes cluster hosted services based on the path routing rules defined in an Ingress Object. I’m a big believer in showing technology through examples, so let’s quickly run through an example to show you what’s possible.

Prerequisites

This example assumes you’re running on a local minikube instance, and that you also have kubectl also running. You can run this same example on your favorite cloud provider managed Kubernetes cluster; you’d need to make a few tweaks. You’ll also need Gloo installed. Let’s use Homebrew to set all of this up for us, and then start minikube and install Gloo. It will take a few minutes to download and install everything to your local machine, and get everything started.

brew update
brew cask install minikube
brew install kubectl glooctl curl

minikube start
glooctl install ingress

One more thing before we dive into Ingress objects, let’s set up an example service deployed on Kubernetes that we can reference.

kubectl apply \
  --filename https://raw.githubusercontent.com/solo-io/gloo/master/example/petstore/petstore.yaml

Setting up an Ingress to our example Petstore

Let’s set up an Ingress object that routes all HTTP traffic to our petstore service. To make this a little more exciting and challenging, and who doesn’t like a good tech challenge, let’s also configure a host domain, which will require a little extra curl magic to call correctly on our local Kubernetes cluster. The following Ingress definition will route all requests to http://gloo.example.com to our petstore service listening on port 8080 within our cluster. The petstore service provides some REST functions listening on the query path /api/pets that will return JSON for the inventory of pets in our (small) store.

If you are trying this example in a public cloud Kubernetes instance, you’ll most likely need to configure a Cloud Load Balancer. Make sure you configure that Load Balancer for the service/ingress-proxy running in the gloo-system namespace.

The important details of our example Ingress definition are:

  • Annotation kubernetes.io/ingress.class: gloo which is the standard way to mark an Ingress object as handled by a specific Ingress controller, i.e., Gloo. This requirement will go away soon as we add the ability for Gloo to be the cluster default Ingress controller
  • Path wildcard /.* to indicate that all traffic to http://gloo.example.com is routed to our petstore service
cat <<EOF | kubectl apply --filename -
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: petstore-ingress
 annotations:
    kubernetes.io/ingress.class: gloo
spec:
  rules:
  - host: gloo.example.com
    http:
      paths:
      - path: /.*
        backend:
          serviceName: petstore
          servicePort: 8080
EOF

We can validate that Kubernetes created our Ingress correctly by the following command.

kubectl get ingress petstore-ingress

NAME               HOSTS              ADDRESS   PORTS   AGE
petstore-ingress   gloo.example.com             80      14h

To test we’ll use curl to call our local cluster. Like I said earlier, by defining a host: gloo.example.com in our Ingress, we need to do a little more to call this without doing things with DNS or our local /etc/hosts file. I’m going to use the recent curl --connect-to options, and you can read more about that at the curl man pages.

The glooctl command-line tool helps us get the local host IP and port for the proxy with the glooctl proxy address --name <ingress name> --port http command. It returns the address (host IP:port) to the Gloo Ingress proxy that allows us external access to our local Kuberbetes cluster. If you are trying this example in a public cloud managed Kuberbetes, then most will handle the DNS mapping for your specified domain (that you should own), and the Gloo Ingress service, so in that case, you do NOT need the --connect-to magic, just curl http://gloo.example.com/api/pets should work.

curl --connect-to gloo.example.com:80:$(glooctl proxy address --name ingress-proxy --port http) \
    http://gloo.example.com/api/pets

Which should return the following JSON

[{"id":1,"name":"Dog","status":"available"},{"id":2,"name":"Cat","status":"pending"}]

TLS Configuration

These days, most want to use TLS to secure your communications. Gloo Ingress can act as a TLS terminator, and we’ll quickly run through what that set up would look like.

Any Kubernetes Ingress doing TLS will need a Kubernetes TLS secret created, so let’s create a self-signed certificate we can use for our example gloo.example.com domain. The following two commands will produce a certificate and generate a TLS secret named my-tls-secret in minikube.

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout my_key.key -out my_cert.cert -subj "/CN=gloo.example.com/O=gloo.example.com"

kubectl create secret tls my-tls-secret --key my_key.key --cert my_cert.cert

Now let’s update our Ingress object with the needed TLS configuration. Important that the TLS host and the rules host match, and the secretName matches the name of the Kubernetes secret deployed previously.

cat <<EOF | kubectl apply --filename -
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: petstore-ingress
  annotations:
    kubernetes.io/ingress.class: gloo
spec:
  tls:
  - hosts:
    - gloo.example.com
    secretName: my-tls-secret
  rules:
  - host: gloo.example.com
    http:
      paths:
      - path: /.*
        backend:
          serviceName: petstore
          servicePort: 8080
EOF

If all went well, we should have changed our petstore to now be listening to https://gloo.example.com. Let’s try it, again using our curl magic, which we need to both resolve the host and port as well as to validate our certificate. Notice that we’re asking glooctl for --port https this time, and we’re curling https://gloo.example.com on port 443. We’ll also have curl validate our TLS certificate using curl --cacert <my_cert.cert> with the certificate we created and used in our Kubernetes secret.

curl --cacert my_cert.cert \
    --connect-to gloo.example.com:443:$(glooctl proxy address --name ingress-proxy --port https) \
    https://gloo.example.com/api/pets

Next Steps

This was a quick tour of how Gloo can act as your Kubernetes Ingress controller making very minimal changes to your existing Kubernetes manifests. Please try it out and let us know what you think at our community Slack channel.

If you’re interested in powering up you Gloo superpowers, try Gloo in gateway mode glooctl install gateway, which unlocks a set of Kubernetes CRDs (Custom Resources) that give you a more standard, and far more powerful, way of doing more advanced traffic shifting, rate limiting, and more without the annotation smell in your Kubernetes cluster. Check out these other articles for more details on Gloo’s extra powers.

Read More