$ oc adm registry --config=/etc/origin/master/admin.kubeconfig \(1)
--service-account=registry \(2)
--images='registry.access.redhat.com/openshift3/ose-${component}:${version}' (3)
If the integrated registry was not previously deployed automatically during the initial installation of your OpenShift Container Platform cluster, or if it is no longer running successfully and you need to redeploy it on your existing cluster, see the following sections for options on deploying a new registry.
|
This topic is not required if you installed a stand-alone registry. |
To deploy the integrated Docker registry, use the oc adm registry command as a
user with cluster administrator privileges. For example:
$ oc adm registry --config=/etc/origin/master/admin.kubeconfig \(1)
--service-account=registry \(2)
--images='registry.access.redhat.com/openshift3/ose-${component}:${version}' (3)
| 1 | --config is the path to the
CLI configuration file for
the cluster
administrator. |
| 2 | --service-account is the service account used to run the registry’s pod. |
| 3 | Required to pull the correct image for OpenShift Container Platform. |
This creates a service and a deployment configuration, both called docker-registry. Once deployed successfully, a pod is created with a name similar to docker-registry-1-cpty9.
To see a full list of options that you can specify when creating the registry:
$ oc adm registry --help
Use the oc adm registry command to deploy the registry as a DaemonSet with the
--daemonset option.
Daemonsets ensure that when nodes are created, they contain copies of a specified pod. When the nodes are removed, the pods are garbage collected.
For more information on DaemonSets, see
Using DaemonSets.
Since there is a plenty of configuration options for deploying the integrated registry, this
section will try to be as generic as possible and will mention exceptions just for the common
cases. Any invocation of oc adm registry command below may be passed additional arguments.
One of the key rules while re-deploying the registry is to never change it’s service IP although it’s very easy to do so.
OpenShift Container Platform refers to the integrated registry by its service IP address unless the internal hostname is configured. When the service IP is used, it’s the admin’s responsibility to preserve it to remain consistent with internal etcd database. If a custom internal hostname is used, it’s still highly recommended to preserve the service IP whenever the registry is re-deployed.
If you decide to delete and recreate the docker-registry service, you can ensure a completely transparent transition by arranging to re-use the old IP address in the new service. If a new IP address cannot be avoided, you can minimize cluster disruption by rebooting only the masters.
The internal registry address is used to construct docker image pull-specifications that are used
by OpenShift Container Platform’s components such us deployer and builder to pull them from the integrated
registry when needed. Moreover, OpenShift Container Platform’s client tools like oc adm prune images
determine the address of the integrated Docker registry from the existing images unless directly
provided.
So if the registry’s service IP does not match with the references in image streams and images, such client tools can misbehave. It also complicates future migration process.
All the example scenarios listed below will stick to this rule.
When only the registry pod specification needs to be changed, all the other registry objects that
are usually created side-by-side with the docker-registry DeploymentConfig or DaemonSet
(deployment from now on) can be left unchaged. Those are:
serviceaccount/registry
service/docker-registry
clusterrolebinding/registry-registry-role
The registry’s behaviour can be easily tuned using environment variables or configuration file as
documented in the
extended
registry configuration. Therefor, there’s no need for deleting the existing deployment. The
admin can utilize tools like oc env, oc volume, or oc edit to change the existing
deployment.
This way, it’s easy to roll-back to older working configurations oc rollout undo ….
In rare cases, it’s possible to create a completely new deployment from scratch. This can be achieved like this:
Back up the existing deployment:
$ oc get -n default -o yaml dc/docker-registry \ (1) >dc-docker-registry.yaml
| 1 | Replace dc with ds if your deployment is a DaemonSet. The same applies to the next
command below. |
Delete the existing deployment:
$ oc delete -n default dc/docker-registry
Create a new deployment with oc adm registry …:
The command will fail because some objects already exist as the following output illustrates. That can be ignored because the desired DeploymentConfig has been successfully created.
$ oc adm registry -o json \ (1) | oc create -f - deploymentconfig "docker-registry" created Error from server (AlreadyExists): serviceaccounts "registry" already exists Error from server (AlreadyExists): clusterrolebindings "registry-registry-role" already exists
| 1 | Add additional arbitrary arguments as you see fit. Add --daemonset if you want to deploy
a DaemonSet. |
Should be avoided if possible. However, it can be useful when the service has been already unintentionally reinstantiated with a different IP while there are existing objects in the registry. The process is different for the DeploymentConfig and DaemonSet.
Replace the existing service object with new one with the desired cluster IP:
$ oc expose -n default dc/docker-registry \ (1) --session-affinity=ClientIP --cluster-ip=172.30.1.1 <2> \ -o json --dry-run | \ oc replace -n default -f - --force (3)
Restart the registry pods:
This is needed unless REGISTRY_OPENSHIFT_SERVER_ADDR is set for the registry services to
reload their internal registry address.
$ oc rollout latest -n default dc/docker-registry
| 1 | The command produces a specification of the new service named docker-registry in the default
namespace for the given DeploymentConfig. |
| 2 | Use the desired IP address instead. |
| 3 | If the service does not exist anymore, change the last line to oc create -f -. |
In case of a DaemonSet, its update strategy needs to be first changed to RollingUpdate so that
pods will be cleanly restarted in a controlled fashion.
Set the update strategy to RollingUpdate:
$ oc patch -n default ds/docker-registry \
-p '{"spec":{"updateStrategy":{"type":"RollingUpdate"}}}'
Replace the registry’s service:
$ oc create service clusterip -n default docker-registry --dry-run --tcp=5000:5000 \
-o json --clusterip=172.30.1.1 | \ (1)
oc patch -f - --type=json -p '[
{"op":"add","path":"/spec/sessionAffinity","value":"ClientIP"},
{"op":"add","path":"/spec/selector","value":{"docker-registry":"default"}}
]' --local=true -o json | \
oc replace -n default -f - --force (2)
Redeploy the DaemonSet pods in a safe manner by updating the pod spec template:
This is needed unless REGISTRY_OPENSHIFT_SERVER_ADDR is set for the registry services to
reload their internal registry address.
$ oc env -n default ds/docker-registry "TIMESTAMP=$(date +%s%N)"
| 1 | Make sure to choose the right IP address. |
| 2 | If the service does not exist anymore, change the last line to oc create -f -. |
The masters cache the internal registry address for their whole lifetime. Unless a custom internal registry hostname is set, the masters must be restarted to minimize cluster’s disruption:
# systemctl restart atomic-openshift-master-api atomic-openshift-master-controllers
This ensures that the old registry URL, which includes the old IP address, is cleared from the cache.
|
We recommend against rebooting the entire cluster because that incurs unnecessary downtime for pods and does not actually clear the cache. |
By default, the registry is created with no settings for compute resource requests or limits. For production, it is highly recommended that the deployment configuration for the registry be updated to set resource requests and limits for the registry pod. Otherwise, the registry pod will be considered a BestEffort pod.
See Compute Resources for more information on configuring requests and limits.
The registry stores container images and metadata. If you simply deploy a pod with the registry, it uses an ephemeral volume that is destroyed if the pod exits. Any images anyone has built or pushed into the registry would disappear.
This section lists the supported registry storage drivers. See the Docker registry documentation for more information.
The following list includes storage drivers that need to be configured in the registry’s configuration file:
Filesystem. Filesystem is the default and does not need to be configured.
S3. See the CloudFront configuration documentation for more information.
General registry storage configuration options are supported. See the Docker registry documentation for more information.
The following storage options need to be configured through the filesystem driver:
|
For more information on supported persistent storage drivers, see Configuring Persistent Storage and Persistent Storage Examples. |
For production use, attach a remote volume or define and use the persistent storage method of your choice.
For example, to use an existing persistent volume claim:
$ oc volume deploymentconfigs/docker-registry --add --name=registry-storage -t pvc \
--claim-name=<pvc_name> --overwrite
|
See Known Issues if using a scaled registry with a shared NFS volume. |
There is also an option to use Amazon Simple Storage Service storage with the internal Docker registry. It is a secure cloud storage manageable through AWS Management Console. To use it, the registry’s configuration file must be manually edited and mounted to the registry pod. However, before you start with the configuration, look at upstream’s recommended steps.
Take a default YAML configuration file as a base and replace the filesystem entry in the storage section with s3 entry such as below. The resulting storage section may look like this:
storage:
cache:
layerinfo: inmemory
delete:
enabled: true
s3:
accesskey: awsaccesskey (1)
secretkey: awssecretkey (2)
region: us-west-1
regionendpoint: http://myobjects.local
bucket: bucketname
encrypt: true
keyid: mykeyid
secure: true
v4auth: false
chunksize: 5242880
rootdirectory: /s3/object/name/prefix| 1 | Replace with your Amazon access key. |
| 2 | Replace with your Amazon secret key. |
All of the s3 configuration options are documented in upstream’s driver reference documentation.
Overriding the registry configuration will take you through the additional steps on mounting the configuration file into pod.
|
When the registry runs on the S3 storage back-end, there are reported issues. |
For non-production use, you can use the --mount-host=<path> option to specify
a directory for the registry to use for persistent storage. The registry volume
is then created as a host-mount at the specified <path>.
|
The |
The --mount-host option requires that the registry container run in privileged
mode. This is automatically enabled when you specify --mount-host.
However, not all pods are allowed to run
privileged containers by default.
If you still want to use this option, create the registry and specify that it use the registry service account that was created during installation:
$ oc adm registry --service-account=registry \
--config=/etc/origin/master/admin.kubeconfig \
--images='registry.access.redhat.com/openshift3/ose-${component}:${version}' \
--mount-host=<path>
|
The Docker registry pod runs as user 1001. This user must be able to write to the host directory. You may need to change directory ownership to user ID 1001 with this command: $ sudo chown 1001:root <path> |
OpenShift Container Platform provides a web-based interface to the integrated registry. This registry console is an optional component for browsing and managing images. It is deployed as a stateless service running as a pod.
|
If you installed OpenShift Container Platform as a stand-alone registry, the registry console is already deployed and secured automatically during installation. |
|
If Cockpit is already running, you’ll need to shut it down before proceeding in order to avoid a port conflict (9090 by default) with the registry console. |
|
You must first have exposed the registry. |
Create a passthrough route in the default project. You will need this when creating the registry console application in the next step.
$ oc create route passthrough --service registry-console \
--port registry-console \
-n default
Deploy the registry console application. Replace <openshift_oauth_url> with
the URL of the OpenShift Container Platform OAuth provider, which is typically the master.
$ oc new-app -n default --template=registry-console \
-p OPENSHIFT_OAUTH_PROVIDER_URL="https://<openshift_oauth_url>:8443" \
-p REGISTRY_HOST=$(oc get route docker-registry -n default --template='{{ .spec.host }}') \
-p COCKPIT_KUBE_URL=$(oc get route registry-console -n default --template='https://{{ .spec.host }}')
|
If the redirection URL is wrong when you are trying to log in to the
registry console, check your OAuth client with |
Finally, use a web browser to view the console using the route URI.
By default, the registry console generates self-signed TLS certificates if deployed manually per the steps in Deploying the Registry Console. See Troubleshooting the Registry Console for more information.
Use the following steps to add your organization’s signed certificates as a
secret volume. This assumes your certificates are available on the oc
client host.
Create a .cert file containing the certificate and key. Format the file with:
One or more BEGIN CERTIFICATE blocks for the server certificate and the intermediate certificate authorities
A block containing a BEGIN PRIVATE KEY or similar for the key. The key must not be encrypted
For example:
-----BEGIN CERTIFICATE----- MIIDUzCCAjugAwIBAgIJAPXW+CuNYS6QMA0GCSqGSIb3DQEBCwUAMD8xKTAnBgNV BAoMIGI0OGE2NGNkNmMwNTQ1YThhZTgxOTEzZDE5YmJjMmRjMRIwEAYDVQQDDAls ... -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIIDUzCCAjugAwIBAgIJAPXW+CuNYS6QMA0GCSqGSIb3DQEBCwUAMD8xKTAnBgNV BAoMIGI0OGE2NGNkNmMwNTQ1YThhZTgxOTEzZDE5YmJjMmRjMRIwEAYDVQQDDAls ... -----END CERTIFICATE----- -----BEGIN PRIVATE KEY----- MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQCyOJ5garOYw0sm 8TBCDSqQ/H1awGMzDYdB11xuHHsxYS2VepPMzMzryHR137I4dGFLhvdTvJUH8lUS ... -----END PRIVATE KEY-----
The registry console loads a certificate from the /etc/cockpit/ws-certs.d directory. It uses the last file with a .cert extension in alphabetical order. Therefore, the .cert file should contain at least two PEM blocks formatted in the OpenSSL style.
If no certificate is found, a self-signed certificate is created using the
openssl command and stored in the 0-self-signed.cert file.
Create the secret:
$ oc secrets new console-secret \
/path/to/console.cert
Add the secrets to the registry-console deployment configuration:
$ oc volume dc/registry-console --add --type=secret \
--secret-name=console-secret -m /etc/cockpit/ws-certs.d
This triggers a new deployment of the registry console to include your signed certificates.
The registry console debug mode is enabled using an environment variable. The following command redeploys the registry console in debug mode:
$ oc set env dc registry-console G_MESSAGES_DEBUG=cockpit-ws,cockpit-wrapper
Enabling debug mode allows more verbose logging to appear in the registry console’s pod logs.
To check which certificate the registry console is using, a command can be run from inside the console pod.
List the pods in the default project and find the registry console’s pod name:
$ oc get pods -n default NAME READY STATUS RESTARTS AGE registry-console-1-rssrw 1/1 Running 0 1d
Using the pod name from the previous command, get the certificate path that the cockpit-ws process is using. This example shows the console using the auto-generated certificate:
$ oc exec registry-console-1-rssrw remotectl certificate certificate: /etc/cockpit/ws-certs.d/0-self-signed.cert