Skip to content

Openshift in Docker on Mac (may2017)

Jump to bottom
Ben Petersen edited this page May 31, 2017 · 6 revisions

Setting up a dev environment using Docker for Mac and oc cluster up

Install Docker for Mac

  1. Install Docker for Mac v1.12.1 making sure you meet the [prerequisites](https://docs.docker. com/docker-for-mac/#/what-to-know-before-you-install).

    Do not update Docker past v1.12.x as there are known compatibility issues

  1. For reference, docker version links (plz use above recommended version, however):

    1. Docker for Mac v1.13.1
    2. Docker for Mac v1.12.1

  2. Once Docker is running, add an insecure registry of 172.30.0.0/16:

    1. From the Docker menu in the toolbar, select Preferences...
    2. Click on Daemon in the preferences dialog
    3. Under Insecure registries:, click on the + icon to add a new entry
    4. Enter 172.30.0.0/16 and press return
    5. Click on Apply and Restart

  3. You will also want to increase the memory and CPU you give Docker

    1. From the Docker menu in the toolbar, select Preferences...
    2. Click on the Advanced tab
    3. Set CPUs to something between 4 and 8
    4. Set memory to at least 6 or 8gig (esp if you have metrics running, else you may be able to get by with less)
    5. Click on Apply and Restart

  4. Install socat

    1. Socat is a command line utility that allows for bidirectional data transfer.

    2. If not already installed, install Homebrew for Mac

    3. Install socat

      Open Terminal and run:

      $ brew install socat

Install the OC binary

  1. Install the oc binary

    • by building oc locally (recommended so you have the latest)

      1. Within your origin fork, run

        hack/build-go.sh cmd/oc

        Note: go (and a GOPATH) is required to run the build-go.sh command.

      2. Determine path to oc binary built in the previous step (e.g., ~/go/src/github.com/openshift/origin/_output/local/bin/darwin/amd64)

  # if you previously installed oc, check where:
  $ which oc   # /usr/local/bin/oc
  # remove it
  $ rm /usr/local/bin/oc
  # and copy the newly built oc to the prev oc's location:
  $ cp ~/go/src/github.com/openshift/origin/_output/local/bin/darwin/amd64/oc /usr/local/bin/oc
  # or, do the following & just add it to your path so you can rebuild whenever you feel like it & not have to 
  # move the file around
  $ sudo nano ~/.bash_profile
  # Add the following, using your correct `oc` binary path
  $ export PATH=$PATH:~/go/src/github.com/openshift/origin/_output/local/bin/darwin/amd64

  • OR using homebrew with: brew install openshift-cli

  • OR by downloading the Mac OS oc binary from openshift-origin-client-tools-VERSION-mac.zip and place it in your path.

    Please be aware that the 'oc cluster' set of commands are only available in the 1.3+ or newer releases.

Cluster up

  1. Open a new Terminal window (so the changes from the previous step are loaded) and run
 # the --use-existing-config flag is optional
 # --version=latest is ideal, but we find that in dev things break
 # --use-existing-config is essential between cluster down / cluster up, if you want to save state
 # --host-data-dir is helpful so you can manipulate files between cluster down / cluster up
 # at times a specific version may be helpful --version=v3.6.0-alpha.1
 # leave off this flag --host-pv-dir=$HOME/go/data/openshift.local.volumes
 $  oc cluster up --version=latest --use-existing-config --host-config-dir=$HOME/go/data/openshift.local.config --host-data-dir=$HOME/go/data/openshift.local.etcd  
 #
 # Be mindful of output, you will need this info later to ensure the web console can talk to the server:
 # 
 # The server is accessible via web console at:
 #    https://127.0.0.1:8443
 #
 # The metrics service is available at:
 #    https://metrics-openshift-infra.127.0.0.1.nip.io 
 #
 # You are logged in as:
 #    User:     developer
 #
 # To login as administrator:
 #    oc login -u system:admin

To stop your cluster, run:

 $ oc cluster down

Running the Catalog & Broker (deprecated)

Don't bother with this, the enableTemplateBroker flag no longer exists in master-config.yaml. We will just install the ups-broker at a later point in these instructions.

# DEPRECATED
# create directories for persistence
$ mkdir ~/go/data/openshift.local.{config,volumes,etcd}
# cluster up for defaults (we will then cluster down & edit)
$ oc cluster up --version=v3.6.0-alpha.1 --host-config-dir=$HOME/go/data/openshift.local.config --host-data-dir=$HOME/go/data/openshift.local.etcd --host-pv-dir=$HOME/go/data/openshift.local.volumes 
# cluster down
$ oc cluster down
# edit the config file, updating enableTemplateServiceBroker from false to true:
$ vim ~/go/data/openshift.local.config/master/master-config.yaml  
# then, cluster it back up
# at times a specific version may be helpful --version=v3.6.0-alpha.1
# leave off this flag --host-pv-dir=$HOME/go/data/openshift.local.volumes
$  oc cluster up --version=latest --use-existing-config --host-config-dir=$HOME/go/data/openshift.local.config --host-data-dir=$HOME/go/data/openshift.local.etcd  
# 
# NOTE:
# the flag --routing-suffix=10.245.2.2.nip.io
# can be used as a hack to fix metrics if you have issues
# if it does not, then you will have to edit your local /etc/hosts file to help overcome this flaw
#
# NOTE: 
# --version=latest is likely sufficient, unless you really need to pin to a particular commit, then:
# --version="$(git log -1 --pretty=%h )"

Now, some evil and dangerous stuff

Allow system:unauthenticated access to the template broker. Yup. Bad. But good, for dev, but bad.

# not in a real env else you be crazy
$ oc login -u system:admin
$ oc adm policy reconcile-cluster-roles --confirm
$ oc adm policy add-cluster-role-to-group templateservicebroker-client system:unauthenticated

Create a Service Catalog Project

$ oc login -u developer -p developer
$ oc new-project service-catalog

Deploy the Catalog Template

Deploy the catalog template

Use this file to deploy the service catalog, either via the web console, or oc create -f with the raw version.

# NOTE: be sure to use a project named `service-catalog`, the api server name will be at
# apiserver-<project-name>.nip.io
$ oc process -f https://gist.githubusercontent.com/jwforres/78d8c2a939e5e69e31ddd32471ce79fd/raw/518ffcd3139671f04aad9f21342f845c992f5543/gistfile1.txt | oc create -f -

Now, double check that things are running.

Run the Web Console

  • visit the web console
  • if its not, then ensure your console config is setup correctly. Here is an example with some comments:
// from the usual directory, HOWEVER, you will need to tinker with your 
// config.local.json. I find it easier to swap your host like this:
//var hostIP = '127.0.0.1';       // localhost
var hostIP = '10.245.2.2';        // => if Vagrant
// var hostIP = '172.30.0.0/16'   // => if Docker
var hostPort = ':8443';
var fullHost = hostIP + hostPort;

  window.OPENSHIFT_CONFIG = {
    apis: {
      hostPort: fullHost,
      prefix: "/apis"
    },
    api: {
      openshift: {
        hostPort: fullHost,
        prefix: "/oapi"
      },
      k8s: {
        hostPort: fullHost,
        prefix: "/api"
      }
    },
    auth: {
      oauth_authorize_uri: 'https://' + fullHost + "/oauth/authorize",
      oauth_redirect_base: "https://localhost:9000/dev-console",
      oauth_client_id: "openshift-web-console",
      logout_uri: ""
    },
    loggingURL: "",
    metricsURL: "https://metrics-openshift-infra." + hostIP + ".nip.io/hawkular/metrics",
    // The additional servers stanza is essential.
    //
    additionalServers: [{
      protocol: "https",
      hostPort: "apiserver-service-catalog." + hostIP + ".nip.io",
      // or this, if you can't get the --routing-suffix=10.245.2.2.nip.io to work properly:
      // if you use this host port, then you need to add a line or two in your /etc/hosts file 
      // on your local machine (not the vagrant machine).  
      // instructions are below 
      hostPort: "apiserver-service-catalog." + '127.0.0.1' + ".nip.io",
      prefix: "/apis"
    }]
  };

  window.OPENSHIFT_VERSION = {
    openshift: "dev-mode",
    kubernetes: "dev-mode"
  };

Once console is ok, check that you can hit the catalog directly (?) https://apiserver-service-catalog.10.245.2.2.nip.io/.

If you can't hit the catalog properly, or if it is listed with a 127.0.0.1 IP address for the route, then edit your /etc/hosts file:

# /etc/hosts
# service catalog stuff
10.245.2.2 apiserver-service-catalog.10.245.2.2.nip.io
10.245.2.2 apiserver-service-catalog.127.0.0.1.nip.io

Perms so the catalog can talk to the API

oc login -u system:admin
oc adm policy add-cluster-role-to-user admin system:serviceaccount:service-catalog:default

Short term: use the UPS broker

The 3 components for the broker are in yaml files in this gist.

# create a project for the broker
$ oc login -u developer -p developer
$ oc new-project ups-broker
# deploy the ups broker, 3 parts, in this order.
# the first two can be done from the CLI,
# the last part should be done from the web console.
$ oc create -f https://gist.githubusercontent.com/spadgett/80f844c380b94e0adacfc614013bc774/raw/9e88765d93f5d435c1abc9fad2cda6dea4562ed9/ups-broker-deployment.yaml -n ups-broker
# deployment "ups-broker" created
$ oc create -f https://gist.githubusercontent.com/spadgett/80f844c380b94e0adacfc614013bc774/raw/9e88765d93f5d435c1abc9fad2cda6dea4562ed9/ups-broker-service.yaml -n ups-broker

Deploy the broker

Finally, in any project in the console (the broker is cluster scoped, so it doesn't matter), navigate to the create 'from yaml' tab and paste in the following (or get it from this gist ups-broker-broker.yaml)

# Step 3
# Run the above 2 commands first
# It is recommended to get this YAML from the actual GIST in case something is updated
# Finally, paste it in the YAML tab of the web console, do not create -f from the CLI, 
# currently openshift does not recognize the kind Broker. :/
apiVersion: servicecatalog.k8s.io/v1alpha1
kind: Broker
metadata:
  name: ups-broker
spec:
  url: http://ups-broker.ups-broker.svc.cluster.local

Then, navigate to the service-catalog project in the console. Go to the controller manager (deployment), check its pod, look at the logs and look for these lines:

1 controller.go:189] Successfully converted catalog payload from Broker ups-broker to service-catalog API
1 controller.go:192] Reconciling serviceClass user-provided-service (broker ups-broker)
1 controller.go:199] Reconciled serviceClass user-provided-service (broker ups-broker)

Tips, Tricks, Bugs, Whatever

Quick rebuild

# in openshift/origin
# quick rebuild the oc command
# if you are running in a container, don't need to compile everything!
$ hack/build-go.sh cmd/oc
# once build, verify which you are using:
$ which oc
# if it isn't something like:
#  _output/local/bin/darwin/amd64
# you prob area not actually using what you built

Run latest origin in a container

$ oc cluster down
$ docker pull openshift/origin:latest
# add whatever other flags you like, just remember that if there were any major changes,
# you may not be able to --use-existing-config
$ oc cluster up --version=latest

Build oc in a container (if you don't have go installed locally)

This will do the build within a container that contains our build runtime and extract oc back out of it:

# because, who builds locally when you can build in a container...srsly & fo'realz
OS_DEBUG=true OS_BUILD_ENV_PRESERVE=_output/local/bin hack/env OS_BUILD_PLATFORMS=darwin/amd64 hack/build-go.sh cmd/oc
Clone this wiki locally