Downloading And Applying our patches to Cloud Control ( Part 2 - PSU )

Using the WGET method download and unzip patch 19218456

Unzip it and run the analyze phase, with the credential key file we have created :
[oracle@ebscloud 19218456]$ /u01/oracle/oms12104/oms/OPatch/opatchauto apply -analyze -property_file /home/oracle/patches/wlsprops.txt
OPatch Automation Tool
Copyright (c) 2014, Oracle Corporation.  All rights reserved.


OPatchauto version : 11.1.0.12.3
OUI version        : 11.1.0.12.0
Running from       : /u01/oracle/oms12104/oms
Log file location  : /u01/oracle/oms12104/oms/cfgtoollogs/opatch/opatch2015-01-14_07-18-43AM_1.log

OPatchauto log file: /u01/oracle/oms12104/oms/cfgtoollogs/opatchauto/19218456/opatch_oms_2015-01-14_07-18-45AM_analyze.log



Configuration Validation: Success


Running apply prerequisite checks for sub-patch(es) "19218456" and Oracle Home "/u01/oracle/oms12104/oms"...
Please monitor OPatch log file: /u01/oracle/oms12104/oms/cfgtoollogs/opatch/19218456_Jan_14_2015_07_18_43/ApplyPrereq2015-01-14_07-19-00AM_2.log
Sub-patch(es) "19218456" are successfully analyzed for Oracle Home "/u01/oracle/oms12104/oms"


Complete Summary
================


All log file names referenced below can be accessed from the directory "/u01/oracle/oms12104/oms/cfgtoollogs/opatch/2015-01-14_07-18-43AM_SystemPatch_19218456_1"

Prerequisites analysis summary:
-------------------------------

The following sub-patch(es) are applicable:

  Oracle Home Name   Sub-patches                                          Log file
  ----------------   -----------                                          --------
           oms12c2      19218456   19218456_ApplyPrereq2015-01-14_07-19-00AM_2.log



Log file location: /u01/oracle/oms12104/oms/cfgtoollogs/opatchauto/19218456/opatch_oms_2015-01-14_07-18-45AM_analyze.log

OPatchauto succeeded.

So now we have validated our patchset, it is now time to apply it per the notes : 

emctl stop oms

Oracle Enterprise Manager Cloud Control 12c Release 4

Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.

Stopping WebTier...

WebTier Successfully Stopped

Stopping Oracle Management Server...

Oracle Management Server Successfully Stopped

Oracle Management Server is Down

[oracle@ebscloud 19218456]$
[oracle@ebscloud 19218456]$ /u01/oracle/oms12104/oms/OPatch/opatchauto apply -property_file /home/oracle/patches/wlsprops.txt
OPatch Automation Tool
Copyright (c) 2014, Oracle Corporation.  All rights reserved.


OPatchauto version : 11.1.0.12.3
OUI version        : 11.1.0.12.0
Running from       : /u01/oracle/oms12104/oms
Log file location  : /u01/oracle/oms12104/oms/cfgtoollogs/opatch/opatch2015-01-17_07-56-03AM_1.log

OPatchauto log file: /u01/oracle/oms12104/oms/cfgtoollogs/opatchauto/19218456/opatch_oms_2015-01-17_07-56-06AM_deploy.log



Configuration Validation: Success


Running apply prerequisite checks for sub-patch(es) "19218456" and Oracle Home "/u01/oracle/oms12104/oms"...
Please monitor OPatch log file: /u01/oracle/oms12104/oms/cfgtoollogs/opatch/19218456_Jan_17_2015_07_56_03/ApplyPrereq2015-01-17_07-56-18AM_2.log
Sub-patch(es) "19218456" are successfully analyzed for Oracle Home "/u01/oracle/oms12104/oms"

To continue, OPatchauto will do the following:
[Patch and deploy artifacts]   : Apply sub-patch(es) [ 19218456 ] to Oracle Home "/u01/oracle/oms12104/oms";
                                 Apply RCU artifact with patch "/u01/oracle/oms12104/oms/.patch_storage/19218456_Oct_14_2014_03_22_39/original_patch"


Do you want to proceed? [y|n]
y
User Responded with: Y

Applying sub-patch "19218456" to Oracle Home "/u01/oracle/oms12104/oms"...
Please monitor OPatch log file: /u01/oracle/oms12104/oms/cfgtoollogs/opatch/19218456_Jan_17_2015_07_56_03/apply2015-01-17_07-56-30AM_4.log

Updating repository with RCU reference file "/u01/oracle/oms12104/oms/.patch_storage/19218456_Oct_14_2014_03_22_39/original_patch"

Complete Summary
================


All log file names referenced below can be accessed from the directory "/u01/oracle/oms12104/oms/cfgtoollogs/opatch/2015-01-17_07-56-03AM_SystemPatch_19218456_1"

Patching summary:
-----------------

Binaries of the following sub-patch(es) have been applied successfully:

  Oracle Home Name   Sub-patches                                    Log file
  ----------------   -----------                                    --------
           oms12c2      19218456   19218456_apply2015-01-17_07-56-30AM_4.log

Deployment summary:
-------------------

The following artifact(s) have been successfully deployed:

  Artifacts                                                  Log file
  ---------                                                  --------
        SQL   rcu_applypatch_original_patch_2015-01-17_07-58-05AM.log


Log file location: /u01/oracle/oms12104/oms/cfgtoollogs/opatchauto/19218456/opatch_oms_2015-01-17_07-56-06AM_deploy.log

OPatchauto succeeded.

And there we are.... 12.1.0.4 PSU applied ! 

Now start it up again with emctl start oms ( don't also forget to do emcli sync on your emcli installs ). 

Setting up opatchauto credential files

In order to have to avoid setting in passwords every time we do opatchauto we can build a credential file :

[oracle@ebscloud patches]$ /u01/oracle/oms12104/oms/OPatch/wlskeys/createkeys.sh -h $ORACLE_HOME -location .
Please enter weblogic admin server username: weblogic
Please enter weblogic admin server password:
Creating the key file can reduce the security of your system if it is not kept in a secured location after it is created. Creating new key...

Trying to get configuration and key files for the given inputs...
This operation will take some time. Please wait for updates...
User configuration file created: /home/oracle/patches/config
User key file created: /home/oracle/patches/key
'createKeys' succeeded.

Now create the small file needed the property file to point at these, lets call it wlsprops

You need the admin server URL for the first one, you can get this from emctl status oms -details and replace https with t3s 

AdminServerURL=t3s://ebscloud.bcs.com:7103

AdminConfigFile=/home/oracle/patches/config

AdminKeyFile=/home/oracle/patches/key

So this can now be used with our Enterprise Manager patching

Updating to latest opatch

In previous post we showed how to get latest opatch,  now we show how to install it


OPATCH before

[oracle@ebscloud ~]$ /u01/oracle/oms12104/oms/OPatch/opatch lsinventory
Oracle Interim Patch Installer version 11.1.0.10.4
Copyright (c) 2015, Oracle Corporation.  All rights reserved.

OPatch is just unzip the 688 zip file straight over the Oracle Home :

set ORACLE_HOME correctly  and

[oracle@ebscloud patches]$ unzip -d $ORACLE_HOME p6880880_111000_Linux-x86-64.zip
Archive:  p6880880_111000_Linux-x86-64.zip
  inflating: /u01/oracle/oms12104/oms/OPatch/wlskeys/README.txt
replace /u01/oracle/oms12104/oms/OPatch/wlskeys/createkeys.cmd? [y]es, [n]o, [A]ll, [N]one, [r]ename:

just press A and ENTER 

....

  creating: /u01/oracle/oms12104/oms/OPatch/ocm/bin/

  inflating: /u01/oracle/oms12104/oms/OPatch/ocm/bin/emocmrsp

  inflating: /u01/oracle/oms12104/oms/OPatch/opatch.bat

  inflating: /u01/oracle/oms12104/oms/OPatch/operr.bat

  inflating: /u01/oracle/oms12104/oms/OPatch/opatchauto.bat

all done.   

OPATCH after 

[oracle@ebscloud patches]$ /u01/oracle/oms12104/oms/OPatch/opatch lsinventory

Oracle Interim Patch Installer version 11.1.0.12.3

Copyright (c) 2015, Oracle Corporation.  All rights reserved.

Using Wget to get Oracle Patches

A little known metalink feature is using wget to download patches.

This can help tremendously in reducing the amount of crud you have to move around from desktop to server as patches can be very big, and people's desktop connections very slow indeed.

The usual patch download process is to find the patch in metalink, then download to the PC, then FTP or SFTP to the server to be patched.

Using Wget lets us download the file direct from metalink to the server in question.

So how to do this wget step ?


Let's assume we start both from the same point :

Here we are downloading opatch, a fairly common patching tool

If I click on the link we will have the patch downloaded to my PC.

But in the bottom we can see WGET options.  Let's look at that :

It tells us we will download a little script that will actually get the files for us, rather than the large download.  This sounds nice.  Let's do that so press the button. 

What we're seeing is windows 8.1 downloading the file to the desktop and asking if we want to open or save it. Notepad is not good with shell scripts cos it doesn't do CRLFs very well so we will save it instead ! 

When we open that file with a proper editor we see

Ok so what do we do with this.   On the database server or host which you want to patch, open a vi editor

Now copy the contents of the script into the vi editor :  ctrl-A to select all , paste into the terminal emulator

OK - can we use this :  We still have to edit it to change it for the correct userid and password so go up to the top of the file
Edit the lines SSO_USERNAME and SSO_PASSWORD to be whatever you use for Metalink like :

SSO_USERNAME=dingo@xyz.com
SSO_PASSWORD=swordfish

NOW we can use this ! 

Exit to the shell and chmod +x the script, and execute it.

You may run into the problem or two with wget, which is the mechanism for getting the patch  not working.   When you try and check this you will see what wget cannot connect to updates.oracle.com  .    Deep checking shows that this is because the SSL server cannot seem to handle TLS V1.2  - this is not a client side issue, even recompiling to the latest version of wget will not solve it.   The only way of doing this is to tell the wget client to downgrade its SSL protocol to TLSV1 only  - you can do that by adding --secure-protocol=TLSV1 and then it all works.

[oracle@ebscloud patches]$ sh dl.sh
[oracle@ebscloud patches]$ ls -lrt
total 53400
-rw-r--r-- 1 oracle oinstall 54484029 Nov 28 14:18 p6880880_111000_Linux-x86-64.zip
-rwxr-xr-x 1 oracle oinstall     1964 Jan  9 08:00 dl.sh
-rw-r--r-- 1 oracle oinstall     3329 Jan  9 08:00 download

[oracle@ebscloud patches]$

And now it has downloaded our patch ! 

Adding a RAID volume to Azure Linux VM

Breaking into our short OEM work I find I have to move it to some faster disk as  a single Azure BLOB has a 500 IOPS limit.   So we need to stripe a bunch of disks together.

To do this we first have to create them in Azure Control Panel ( I hate Powershell )

Click Attach Empty Disk at the bottom.   We will do  15x10G disks

]

Repeat till 10 done ! 

Sorry ( don't use Read/Write above , use None or you will run into a problem with it,  that onyl  4 R/W can be there ).



now lets' install mdadm

[root@ebscloud ~]# yum install mdadm
Loaded plugins: refresh-packagekit
public_ol6_addons                                                                                                                                         | 1.2 kB     00:00
public_ol6_addons/primary                                                                                                                                 |  77 kB     00:00
public_ol6_addons                                                                                                                                                        264/264
public_ol6_latest                                                                                                                                         | 1.4 kB     00:00
public_ol6_latest/primary                                                                                                                                 |  45 MB     00:03
public_ol6_latest                                                                                                                                                    29084/29084
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package mdadm.x86_64 0:3.2.6-7.el6_5.2 will be updated
---> Package mdadm.x86_64 0:3.3-6.el6_6.1 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================================================================================================
 Package                              Arch                                  Version                                       Repository                                        Size
=================================================================================================================================================================================
Updating:
 mdadm                                x86_64                                3.3-6.el6_6.1                                 public_ol6_latest                                341 k

Transaction Summary
=================================================================================================================================================================================
Upgrade       1 Package(s)

Total download size: 341 k
Is this ok [y/N]: y
Downloading Packages:
mdadm-3.3-6.el6_6.1.x86_64.rpm                                                                                                                            | 341 kB     00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Updating   : mdadm-3.3-6.el6_6.1.x86_64                                                                                                                                    1/2
  Cleanup    : mdadm-3.2.6-7.el6_5.2.x86_64                                                                                                                                  2/2
  Verifying  : mdadm-3.3-6.el6_6.1.x86_64                                                                                                                                    1/2
  Verifying  : mdadm-3.2.6-7.el6_5.2.x86_64                                                                                                                                  2/2

Updated:
  mdadm.x86_64 0:3.3-6.el6_6.1

Complete!
[root@ebscloud ~]#

[root@ebscloud ~]# fdisk -l|grep 10
Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 26108 cylinders
/dev/sdc1               1       26108   209712478+  83  Linux
Disk /dev/sdd: 10.7 GB, 10737418240 bytes
Disk /dev/sde: 10.7 GB, 10737418240 bytes
Disk /dev/sdg: 10.7 GB, 10737418240 bytes
Disk /dev/sdf: 10.7 GB, 10737418240 bytes
Disk /dev/sdj: 10.7 GB, 10737418240 bytes
Disk /dev/sdi: 10.7 GB, 10737418240 bytes
Disk /dev/sdl: 10.7 GB, 10737418240 bytes
Disk /dev/sdm: 10.7 GB, 10737418240 bytes
Disk /dev/sdk: 10.7 GB, 10737418240 bytes
Disk /dev/sdh: 10.7 GB, 10737418240 bytes
Disk /dev/sdn: 10.7 GB, 10737418240 bytes

[root@ebscloud ~]#

so those are the disks.  Repeat for all 10

[root@ebscloud ~]# fdisk /dev/sdd
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xe1519968.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@ebscloud ~]#


etc 
Now run mdadm --create /dev/md0 --verbose --level 0 --raid-devices 15 /dev/sdd1 /dev/sde1 /dev/sdg1 /dev/sdf1 /dev/sdj1 /dev/sdi1 /dev/sdl1 /dev/sdm1 /dev/sdk1 /dev/sdh1 /dev/sdn1 /dev/sdo1 /dev/sdp1 /dev/sdq1 /dev/sdr1

this will give you a Linux device /dev/md0 

then do mkfs.ext4 on it and mount it in fstab . 

Et Voila now you have a drive.   ( if you've messed up and built Grid Control on a slow disk we will now give you several hours to push it all across to the new drive and then remap drives etc ) 

.. all done..  several Azure features were bumped into like no more than 16 attached disks etc  

Patching the Cloud Control - Part 1 - What Patches to Apply.

In looking at Oracle's fantastic Enterprise Manager Product we should get the usual "Update to Latest Patchsets" out of the way first.  This is relevant because Oracle update OEM very fast ( its better than it used to be so we are taking full advantage ! )


We consider here our repository database to be up-to-date, so updates here refer only to the OMS components

For the Cloud Control How do We Do that ?

Part 1 : OMS

metalink points the way, usually

and a quick search points out

Master Index for Cloud Control Agent Installation, Upgrade and Patching (Doc ID 1363767.1)

and this lovely document points us to 

793512.1  - Release Schedule of Current Enterprise Manager Releases and Patch Sets

Since I am on Linux x86-64 and on 12.1.0.4 this lovely document tells me my OMS is at the latest version. 

But it also links to 822485.1  Oracle Recommended Patches (PSU) for Enterprise Manager Base Platform (All Releases)

And this tells me that the latest patchset ( at time of writing Jan 2015 ) is 

12.1.0.4.1 Enterprise Manager Base Platform Patch Set Update (PSU) which can be found at 

If we look at that we find  

This PSU1 is an OMS patch only.

• Patch 19218456

This PSU1 can be installed on top of Enterprise Manager Base Platform 12.1.0.4.0.

So we'll look at that in our next steps and how to do that.   

But that is the OMS Side Patch 

Agent-Side Patches 

Let's look at 

Enterprise Manager 12.1.0.4.0 (PS3) Master Bundle Patch List for Agent and Plug-ins (OMS and Agent side) (Doc ID 1900943.1)

This tells us we need the patch for 

System Patch 12.1.0.4.7 at Ml Note Patch 20188140

and we need the latest agent patch Patch 19513382

So that should be the set of patches we will need to do.   In the next phase we will show how to do this. 

How to Connect to the Cloud Control

So your fantastic System Administrators have installed the cloud control for you.
In the previous post we showed you how to start the cloud control and check it is up.

Now we can show you how to log on to it !

From the previous post I showed you how to get the URL for the Cloud Control
 https://ebscloud.bsx.com:7803/em

Open a browser and enter your URL into the address bar

now enter the sysman userid = SYSMAN and the assigned password

press Enter and eventually ( can take several errors and retries if it is a slow oms and a slow link )

Note that different home pages can show up if you selected a different one to appear when you first logged in, but its easy to change to whichever one matters to you. 

How to find out what version of cloud control you are running ... Click your username in the top right hand corner, and in the ensuing menu click About Enterprise Manager

If you want to log out of the Cloud Control ( you should always log out as your credentials can be very powerful giving you unlimited access ) 

at the top right corner you can see your username SYSMAN or whatever, click it, then the menu pops out, click  Log Out 

We can see we have been logged out.   IN the next post we will look at patching the Cloud Control to add all the latest updates.

Happy New Year

Chris.