Friday, November 23, 2012

Connecting Oracle Business Intelligence OBIEE to ODBC SQL Server Data Source



Hello again.   One thing I wanted to blog about was to how to get OBIEE to talk to SQL Server.

Version of  OBIEE :  11.1.1.6 ,  OS Linux x86-64

Version of SQL Server :  2008.

There are two things to do, one easy and one a little complicated.

The easy one to do is to connect rhe Admin PC ; this is as simple as creating a SQL Server ODBC DSN in ODBC Administrator.  You can then reference this ODBC DSN  in the Repository Builder.

The only real thing to be noted here is the consistent naming convention that must be adopted; you MUST give the DSN a valid name such as LIVE_DWH , TEST_DWH or some such.  The reason for this is you will be using it on the server side, as the name will be encoded into the RPD and so it will look for it on our ODBC config on the server side.


So we assume when starting here you are configuring the server side only  to talk to the ODBC data source.

I presume a slightly different approach is needed for a Windows server OS but it may not be, I haven't tried.

Items to be obtained before we continue.

1.  IP Address or hostname of the database server.
2. Port Number for the SQL Server Database This is usually 1433

Before you proceed ensure you can reach the database server, e.g. via telnet 1433.  If the screen clears or responds then the commlink is usually OK, if unreachable or timeout then you need to fix before progressing ( usually a firewall issue ). If telnet is not present ask for it to be installed or su - yourself and yum -y install telnet 

So we know we can reach the server.  We now need to know

3.  Userid and password to connect to the database.
4.  Name of the database inside SQL Server.

Merant Drivers. 
Oracle supply the Merant drivers to enable OBIEE to talk to an ODBC data source; specifically what you are doing is enabling nqserver to do so.

Unfortunately to confuse matters they also supply  2 different driver versions ( 5.3 and 6.0 ) and the default 5.3  one doesn't work ( well it doesn't for me )  to talk to SQL Server 2008. However the 6.0 has no issues.

So our story is
1:  Reconfigure OBIEE to use 6.0 drivers
2:  Configure the drivers to point at our DSN
3:  Check functionality.

Reconfiguring OBIEE

OBIEE Siebel processes ( nqserver, sawserver etc ) are executables that are controlled in V11 by the opmn framework.

Environment settings for these processes are contained in opmn.xml


OPMN to use the 6.0 Drivers 

Therefore in order to make OBIEE use the 6.0 drivers instead of the 5.3 ones we must edit opmn,xml.

Our first task is to find the right opmn.xml file

If you've installed OBIEE in the best practice setup you will have two, one for the FMW components and one for the WT components

The one you want is the FMW one.

This is usuallly found in

$OH//mw/instances/instance1/config/OPMN/opmn/opmn.xml

OK.  So take a backup of the file ( if you don't, and it breaks, it is one of the easiest means of making the OBIEE completely unusable as it is a key configuration file. ).

In this file you will see several entries for LD_LIBPATH and PATH.

These paths contain references to the Merant 5.3 drivers -e.g.


   

We need to point to all the 6.0 drivers instead.  Check the driver are installed 

ls $ORACLE_HOME/common/ODBC/Merant




]$ ls -lrt /mw/Oracle_BI1/common/ODBC/Merant/
total 16
 4096 Jun 14  2011 5.3
 4096 Jun 14  2011 5.2
 4096 Jun 14  2011 6.0SP1
 4096 Jun 14  2011 6.0

OK so they are there.
Now we need to replace the paths to 6.0 instead

Do this manually in opmn.xml above

or you can use ( no charge ! )



 sed 's/Merant\/5\.3/Merant\/6\.0/' < ./mw/instances/instance1/config/OPMN/opmn/opmn.xml > opmn.xml.new , and replace the files as appropriate

if you can't be bothered with all that typing 

OBIEE ODBC to point at Sql Server 2008 

 With the above fixes we now have good drivers to use, the next thing to is to configure the drivers to point at our needed DSN.  This means editing ODBC.ini

From the Windows Admin tool you have already added an ODBC DSN . We will call this FOODSN. 

Since the RPD wants this we also create it in ODBC.INI

Where edits are done below to protect the client settings I have **  them. 

[FOODSN]
Driver= ** PATH TO ** /Oracle_BI1/common/ODBC/Merant/6.0/lib/ARmsss24.so
Description=DataDirect 6.0 SQL Server Native Wire Protocol
AlternateServers=
AlwaysReportTriggerResults=0
AnsiNPW=1
ApplicationName=
ApplicationUsingThreads=1
AuthenticationMethod=1
BulkBinaryThreshold=32
BulkCharacterThreshold=-1
BulkLoadBatchSize=1024
BulkLoadOptions=2
ConnectionReset=0
ConnectionRetryCount=0
ConnectionRetryDelay=3
Database=**  NAME OF SQL SERVER DATABASE 
EnableBulkLoad=0
EnableQuotedIdentifiers=0
EncryptionMethod=0
FailoverGranularity=0
FailoverMode=0
FailoverPreconnect=0
FetchTSWTZasTimestamp=0
FetchTWFSasTime=1
GSSClient=native
Address=** IP ADDRESS  OF THE DATABASE, PORT OF THE SQL CONNECTION;
HostNameInCertificate=
InitializationString=
Language=
LoadBalanceTimeout=0
LoadBalancing=0
LoginTimeout=15
LogonID=** USERID TO CONNECT TO SQL SERVER 
MaxPoolSize=100
MinPoolSize=0
PacketSize=-1
Password=** PASSWORD TO CONNECT TO SQL SERVER 
Pooling=0
PortNumber=** WHATEVER THE PORT IS 
QueryTimeout=0
ReportCodePageConversionErrors=0
SnapshotSerializable=0
TrustStore=
TrustStorePassword=
ValidateServerCertificate=1
WorkStationID=
XML Describe Type=-10
Trace = 1
TraceFile = ** PATH TO SOME TRACE FILE IF PROBLEMS OCCUR 
TraceDll =  ** PATH TO ** /Oracle_BI1/common/ODBC/Merant/6.0/lib/odbctrac.so

OK.  To make all this work you must restart OPMN 

opmnctl stopall
opmnctl startall 

For debugging purposes, if errors occur they will be seen when you try and access a table in FOODSN.  

The best source of debugging errors is 
1. nqserver.log
2.  IF you are very lucky the ODBC trace routines will work.  This is very tedious to get working. 



Thursday, November 22, 2012

Changing Discoverer to run in Amazon EC2 AWS

Installing Discoverer in Amazon EC2 on linux is straightforward with only a few problems so we won't go into that here.
What I want to talk about is how to make Discoverer deal with instance restarts. This is a problem because usually in EC2 public when the instance restarts it will get a new hostname, and so Discoverer will stop working.

Disco Version : 10.1.2.3  NOT CHECKED AND PROBABLY DOESN'T work for WLS Disco 11+

So how do we solve this ?

There is a number of things to fix.


  1. Database
  2. Middle Tier
  3. Apache 

The intent is to fix these in a repeatable, scriptable way ; we're not quite there yet but we have simple steps to fix them all; it's just a question of scripting the damn things. 

1. Database :  
Easy - we assume no DB control, Grid Control, or agent.  In this case all that needs fixing is the local hostname in listener.ora 
Use  curl http://169.254.169.254/latest/meta-data/local-hostname
to find out the local hostname and sub it into listener. ora

2. Middle Tier 
We need to run chgiphost.sh. 
Source the AS environment and cd $ORACLE_HOME/chgipost/scripts
./chgiphost.sh 

oracle@ip-foonew scripts]$ ./chgiphost.sh  -mid
Oracle Home set to /u01/oracle/as
Starting Change Hostname/IP...

Enter fully qualified hostname (hostname.domainname) of destination
 (ip-foonew.eu-west-1.compute.internal)>>
Enter fully qualified hostname (hostname.domainname) of source
 (ip-fooold.eu-west-1.compute.internal)>>
Enter valid IP Address of destination
 (foonew)>>
Enter valid IP Address of source
 (fooold)>>
Change Hostname/IP completed successfully.

( In the above foonew and foold stand for the actuals from my environment.  chgipghost is a neat little script that figures out the new and old IP addresses 


3.  Apache. 

 In order to make Disco serve up properly over the Internet we had had to configure Apache to return the correct servername

Now that the name has changed we need to fix it again. 
So we need the existing public hostname

curl http://169.254.169.254/latest/meta-data/public-hostname

ec2-fooexternal.eu-west-1.compute.amazonaws.com

in httpd.conf change 
ServerName ec2foooldexternal.eu-west-1.compute.amazonaws.com

to 
ServerName ec2fooexternal.eu-west-1.compute.amazonaws.com


All should be OK now

startup db and ias 

$ORACLE_HOME/opmn/bin/opmnctl startall

All works  !  Tea and biscuits for everyone 

More later ! 

Friday, November 9, 2012

OEL 5 Repository Configuration.

Oracle Enterprise Linux 5 as shipped isn't connected to the free public yum OEL repositories.

In order to fix this to get system updates and any packages, you need to set Yum up.

This is massively easier than it looks.

Ensure your OEL system can reach the Net e.g. via

wget http://www.google.ie or some such.

We need to get the public repo details so


cd /etc/yum.repos.d
wget http://public-yum.oracle.com/public-yum-el5.repo

It should download the repo file to this directory. 

Assuming you want the latest build of everything, you are done, as the latest repo is by default enabled ; otherwise edit the repo file to enable the version you want 

Then it's just yum update.

All done. 

* Note el5 is different for el4 and el6 of course. 






Tuesday, November 6, 2012

Logging on to the E-Business Suite

http://ec2-foo.compute-1.amazonaws.com:8000
use sysadmin/sysadmin or any of the preconfigured users from 

( end of the page ) 






And there we are !




Configuring the Context for the Correct Hostnames


We need to stop the EBS frontend first.


[oracle@]$ /u01/E-BIZ/inst/apps/VISION_ip/admin/sc                                                                                                             ripts/adstpall.sh apps/apps

You are running adstpall.sh version 120.10.12010000.4

The logfile for this session is located at /u01/E-BIZ/inst/apps/VISION_/logs/appl/admin/log/adstpall.log
Executing service control script:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/jtffmctl.sh stop
script returned:
****************************************************

You are running jtffmctl.sh version 120.3

Shutting down Fulfillment Server for VISION on port 9300 ...

jtffmctl.sh: exiting with status 0


.end std out.

.end err out.

****************************************************

adstpall.sh:Exiting with status 0

adstpall.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_/logs/appl/admin/log/adstpall.log for more information ...

[ Edited to remove output from lots of sub-scripts ] 

Now we can edit the context file

Go to /u01/E-BIZ/inst/apps/VISION_ip-/appl/admin and backup the XML context file.

Now edit it :

s_login_page variable needs to replate the FQDN internal with the FQDN external

http://ip-foo.ec2.internal:8000/OA_HTML/AppsLogin

changes to
ec2-foo2.compute-1.amazonaws.com

http://ec2-foo2.compute-1.amazonaws.com:8000/OA_HTML/AppsLogin


_webentryhost changes from

 ip-foo

to
ec2-foo2

i.e. just the hostname portion NOT the FQDN

also change

ec2.internal

to

compute-1.amazonaws.com

We're not changing the default port but if we were we'd edit the ports as per the note.

Save the changes and exit.

Activating the Context Changes : 


Run $INST_TOP



( /u01/E-BIZ/inst/apps/VISION_foo)/admin/scripts/adautocfg.sh



[oracle@ip-foo admin]$  /u01/E-BIZ/inst/apps/VISION_ip-foo/admin/scripts/adautocfg.sh
Enter the APPS user password:

The log file for this session is located at: /u01/E-BIZ/inst/apps/VISION_ip-foo/admin/log/11060530/adconfig.log

AutoConfig is configuring the Applications environment...

AutoConfig will consider the custom templates if present.
        Using CONFIG_HOME location     : /u01/E-BIZ/inst/apps/VISION_ip-foo
        Classpath                   : /u01/E-BIZ/apps/apps_st/comn/java/lib/appsborg2.zip:/u01/E-BIZ/apps/apps_st/comn/java/classes

        Using Context file          : /u01/E-BIZ/inst/apps/VISION_ip-foo/appl/admin/VISION_ip-foo.xml

Context Value Management will now update the Context file

       Updating Context file...COMPLETED

        Attempting upload of Context file and templates to database...COMPLETED

Configuring templates from all of the product tops...
        Configuring AD_TOP........COMPLETED
        Configuring FND_TOP.......COMPLETED
        Configuring ICX_TOP.......COMPLETED
        Configuring MSC_TOP.......COMPLETED
        Configuring IEO_TOP.......COMPLETED
        Configuring BIS_TOP.......COMPLETED
        Configuring AMS_TOP.......COMPLETED
        Configuring CCT_TOP.......COMPLETED
        Configuring WSH_TOP.......COMPLETED
        Configuring CLN_TOP.......COMPLETED
        Configuring OKE_TOP.......COMPLETED
        Configuring OKL_TOP.......COMPLETED
        Configuring OKS_TOP.......COMPLETED
        Configuring CSF_TOP.......COMPLETED
        Configuring IGS_TOP.......COMPLETED
        Configuring IBY_TOP.......COMPLETED
        Configuring JTF_TOP.......COMPLETED
        Configuring MWA_TOP.......COMPLETED
        Configuring CN_TOP........COMPLETED
        Configuring CSI_TOP.......COMPLETED
        Configuring WIP_TOP.......COMPLETED
        Configuring CSE_TOP.......COMPLETED
        Configuring EAM_TOP.......COMPLETED
        Configuring FTE_TOP.......COMPLETED
        Configuring ONT_TOP.......COMPLETED
        Configuring AR_TOP........COMPLETED
        Configuring AHL_TOP.......COMPLETED
        Configuring OZF_TOP.......COMPLETED
        Configuring IES_TOP.......COMPLETED
        Configuring CSD_TOP.......COMPLETED
        Configuring IGC_TOP.......COMPLETED

AutoConfig completed successfully.
You have new mail in /var/spool/mail/oracle
[oracle@ip-foo admin]$

At this point if required we would sort out the suid stuff for apache but we are on 8000 > 1024 so no need

Now let's start Apps front-end again




[oracle@ip-foo admin]$  /u01/E-BIZ/inst/apps/VISION_ip-foo/admin/scripts/adstrtal.sh

You are running adstrtal.sh version 120.15.12010000.3


Enter the APPS username: apps

Enter the APPS password:
The logfile for this session is located at /u01/E-BIZ/inst/apps/VISION_ip-foo/logs/appl/admin/log/adstrtal.log
Executing service control script:
/u01/E-BIZ/inst/apps/VISION_ip-foo/admin/scripts/adopmnctl.sh start
script returned:
****************************************************

You are running adopmnctl.sh version 120.6.12010000.5

Starting Oracle Process Manager (OPMN) ...
opmnctl: opmn started.

adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_ip-foo/logs/appl/admin/log/adopmnctl.txt for more information ...


.end std out.

.end err out.

****************************************************


Executing service control script:
/u01/E-BIZ/inst/apps/VISION_ip-foo/admin/scripts/adalnctl.sh start
script returned:
****************************************************

adalnctl.sh version 120.3

Checking for FNDFS executable.
Starting listener process APPS_VISION.

adalnctl.sh: exiting with status 0


adalnctl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_ip-foo/logs/appl/admin/log/adalnctl.txt for more information ...


.end std out.

.end err out.

****************************************************


Executing service control script:
/u01/E-BIZ/inst/apps/VISION_ip-foo/admin/scripts/adapcctl.sh start
script returned:
****************************************************

You are running adapcctl.sh version 120.7.12010000.2

Starting OPMN managed Oracle HTTP Server (OHS) instance ...
opmnctl: opmn is already running.
opmnctl: starting opmn managed processes...

adapcctl.sh: exiting with status 0

adapcctl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_ip-foo/logs/appl/admin/log/adapcctl.txt for more information ...


.end std out.

.end err out.

****************************************************


Executing service control script:
/u01/E-BIZ/inst/apps/VISION_ip-foo/admin/scripts/adoacorectl.sh start
script returned:
****************************************************

You are running adoacorectl.sh version 120.13

Starting OPMN managed OACORE OC4J instance  ...

adoacorectl.sh: exiting with status 0

adoacorectl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_ip-foo/logs/appl/admin/log/adoacorectl.txt for more information ...


.end std out.

.end err out.




All enabled services for this node are started.

adstrtal.sh: Exiting with status 0

adstrtal.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_ip-foo/logs/appl/admin/log/adstrtal.log for more information ...

[oracle@ip-foo admin]$


[ Edited to remove a ton of unnecessary subscripts starting up ] 


That should be it ... let's give it a go in the next post. 






Finding the Apache Ports and Hostname for the correct Context.

With the previous config Apps thinks it is living on the Internal EC2 addresses.  We need to tell it to listen and respond on the external IP, otherwise it won't work correctly.

You need the location of the context file created in the previous step.  This is
/u01/E-BIZ/inst/apps/VISION_ip-foo/appl/admin where foo is of course the internal IP

You also need the external IP address.  This will be of form

ec2-foo2.compute-1.amazonaws.com  where foo2 is an IP, again ( different from the foo above )

Do we want the port set ?  doing lsof tells us


 lsof -P |grep IPv4|grep http
httpd      3560    oracle   17u     IPv4             171553                 TCP *:4443 (LISTEN)
httpd      3560    oracle   18u     IPv4             171554                 TCP *:8000 (LISTEN)
httpd      3575    oracle   17u     IPv4             171553                 TCP *:4443 (LISTEN)
httpd      3575    oracle   18u     IPv4             171554                 TCP *:8000 (LISTEN)
httpd      3575    oracle   31u     IPv4             171570                 TCP localhost.localdomain:2473->localhost.localdomain:6100 (ESTABLISHED)
httpd      3577    oracle   17u     IPv4             171553                 TCP *:4443 (LISTEN)
httpd      3577    oracle   18u     IPv4             171554                 TCP *:8000 (LISTEN)
httpd      3578    oracle   17u     IPv4             171553                 TCP *:4443 (LISTEN)
httpd      3578    oracle   18u     IPv4             171554                 TCP *:8000 (LISTEN)
httpd      3579    oracle   17u     IPv4             171553                 TCP *:4443 (LISTEN)
httpd      3579    oracle   18u     IPv4             171554                 TCP *:8000 (LISTEN)
httpd      3582    oracle   17u     IPv4             171553                 TCP *:4443 (LISTEN)
httpd      3582    oracle   18u     IPv4             171554                 TCP *:8000 (LISTEN)
httpd      3583    oracle   17u     IPv4             171553                 TCP *:4443 (LISTEN)
httpd      3583    oracle   18u     IPv4             171554                 TCP *:8000 (LISTEN)

all our apache workers are using 8000 and 4443 and that should be OK ! 




Monday, November 5, 2012

Configuring the E-Business Suite Apps Tier on Amazon

Again Reference Note is 1205963.1 . Most variables here are constant, straight from the Metalink note, without any change.  Exceptions are the Internal IPs for the App and Database Servers, of course; they will be different for each machine. use ifconfig -a etc to find out the correct values.

* su to the oracle user
o su oracle
* change directory to /u01/E-BIZ/apps/apps_st/comn/clone/bin
o cd /u01/E-BIZ/apps/apps_st/comn/clone/bin
* Execute the Apps tier post-clone script
o perl adcfgclone.pl appsTier



Target System Hostname (virtual or normal) [foofoo] :  Should be auto populated
 DB SID VISION

Target System Database Server Node [ip-foofoo]  { from the previous section ]


                     Copyright (c) 2002 Oracle Corporation
                        Redwood Shores, California, USA

                        Oracle Applications Rapid Clone

                                 Version 12.0.0

                      adcfgclone Version 120.31.12010000.8

Enter the APPS password :

Running:
/u01/E-BIZ/apps/apps_st/comn/clone/bin/../jre/bin/java -Xmx600M -cp /u01/E-BIZ/apps/apps_st/comn/clone/jlib/java:/u01/E-BIZ/apps/apps_st/comn/clone/jlib/xmlparserv2.jar:/u01/E-BIZ/apps/apps_st/comn/clone/jlib/ojdbc14.jar oracle.apps.ad.context.CloneContext -e /u01/E-BIZ/apps/apps_st/comn/clone/bin/../context/apps/CTXORIG.xml -validate -pairsfile /tmp/adpairsfile_27197.lst -stage /u01/E-BIZ/apps/apps_st/comn/clone  2> /tmp/adcfgclone_27197.err; echo $? > /tmp/adcfgclone_27197.res

Log file located at /u01/E-BIZ/apps/apps_st/comn/clone/bin/CloneContext_1105094800.log

Provide the values required for creation of the new APPL_TOP Context file.

Target System Hostname (virtual or normal) [as mentioned above ] :

Target System Database SID : VISION


Target System Database Server Node [wrong] : correct

Target System Base Directory :/u01/E-BIZ/apps

Target System Tools ORACLE_HOME Directory [/u01/E-BIZ/apps/apps/tech_st/10.1.2]                                                                                                : /u01/E-BIZ/apps/tech_st/10.1.2

Target System Web ORACLE_HOME Directory [/u01/E-BIZ/apps/apps/tech_st/10.1.3] :/u01/E-BIZ/apps/tech_st/10.1.3

Target System APPL_TOP Directory [/u01/E-BIZ/apps/apps/apps_st/appl] : /u01/E-BIZ/apps/apps_st/appl

Target System COMMON_TOP Directory [/u01/E-BIZ/apps/apps/apps_st/comn] : /u01/E-BIZ/apps/apps_st/comn

Target System Instance Home Directory [/u01/E-BIZ/apps/inst] : /u01/E-BIZ/inst

Target System Root Service [enabled] :


Target System Web Entry Point Services [enabled] :

Target System Web Application Services [enabled] :

Target System Batch Processing Services [enabled] :

Target System Other Services [disabled] :

Do you want to preserve the Display [atgtxk-09:0.0] (y/n)  : n

Target System Display [ip-10-71-85-194:0.0] :

Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n

Target System Port Pool [0-99] : 0

Checking the port pool 0
done: Port Pool 0 is free
Report file located at /u01/E-BIZ/inst/apps/VISION_/admin/out/portpool.lst
Complete port information available at /u01/E-BIZ/inst/apps/VISION_/admin/out/portpool.lst

UTL_FILE_DIR on database tier consists of the following directories.

1. /usr/tmp
2. /usr/tmp
3. /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/outbound/VISION_
4. /usr/tmp
Choose a value which will be set as APPLPTMP value on the target node [1] :
Creating the new APPL_TOP Context file from :
  /u01/E-BIZ/apps/apps_st/appl/ad/12.0.0/admin/template/adxmlctx.tmp

The new APPL_TOP context file has been created :
  /u01/E-BIZ/inst/apps/VISION_/appl/admin/VISION_.xml

Log file located at /u01/E-BIZ/apps/apps_st/comn/clone/bin/CloneContext_1105121147.log
Check Clone Context logfile /u01/E-BIZ/apps/apps_st/comn/clone/bin/CloneContext_1105121147.log for details.

Running Rapid Clone with command:
perl /u01/E-BIZ/apps/apps_st/comn/clone/bin/adclone.pl java=/u01/E-BIZ/apps/apps_st/comn/clone/bin/../jre mode=apply stage=/u01/E-BIZ/apps/apps_st/comn/clone component=appsTier method=CUSTOM appctxtg=/u01/E-BIZ/inst/apps/VISION_appl/admin/VISION_.xml showProgress contextValidated=true
Running:
perl /u01/E-BIZ/apps/apps_st/comn/clone/bin/adclone.pl java=/u01/E-BIZ/apps/apps_st/comn/clone/bin/../jre mode=apply stage=/u01/E-BIZ/apps/apps_st/comn/clone component=appsTier method=CUSTOM appctxtg=/u01/E-BIZ/inst/apps/VISION_/appl/admin/VISION_.xml showProgress contextValidated=true
APPS Password :

Beginning application tier Apply - Mon Nov  5 12:22:19 2012

/u01/E-BIZ/apps/apps_st/comn/clone/bin/../jre/bin/java -Xmx600M -DCONTEXT_VALIDATED=true  -Doracle.installer.oui_loc=/oui -classpath /u01/E-BIZ/apps/apps_st/comn/clone/jlib/xmlparserv2.jar:/u01/E-BIZ/apps/apps_st/comn/clone/jlib/ojdbc14.jar:/u01/E-BIZ/apps/apps_st/comn/clone/jlib/java:/u01/E-BIZ/apps/apps_st/comn/clone/jlib/oui/OraInstaller.jar:/u01/E-BIZ/apps/apps_st/comn/clone/jlib/oui/ewt3.jar:/u01/E-BIZ/apps/apps_st/comn/clone/jlib/oui/share.jar:/u01/E-BIZ/apps/apps_st/comn/clone/jlib/oui/srvm.jar:/u01/E-BIZ/apps/apps_st/comn/clone/jlib/ojmisc.jar  oracle.apps.ad.clone.ApplyAppsTier -e /u01/E-BIZ/inst/apps/VISION_/appl/admin/VISION_.xml -stage /u01/E-BIZ/apps/apps_st/comn/clone    -showProgress
APPS Password : Log file located at /u01/E-BIZ/inst/apps/VISION_194/admin/log/ApplyAppsTier_11051222.log
  |      0% completed
  /     71% completed

Completed Apply...
Mon Nov  5 12:33:22 2012


Do you want to startup the Application Services for VISION? (y/n) [y] :


Starting application Services for VISION:
Running:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/adstrtal.sh -nopromptmsg

You are running adstrtal.sh version 120.15.12010000.3

The logfile for this session is located at /u01/E-BIZ/inst/apps/VISION_/logs/appl/admin/log/adstrtal.log
Executing service control script:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/adopmnctl.sh start
script returned:
****************************************************

You are running adopmnctl.sh version 120.6.12010000.5

Starting Oracle Process Manager (OPMN) ...
opmnctl: opmn started.

adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_/logs/appl/admin/log/adopmnctl.txt for more information ...


.end std out.

.end err out.

****************************************************


Executing service control script:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/adalnctl.sh start
script returned:
****************************************************

adalnctl.sh version 120.3

Checking for FNDFS executable.
Starting listener process APPS_VISION.

adalnctl.sh: exiting with status 0


adalnctl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_/logs/appl/admin/log/adalnctl.txt for more information ...


.end std out.

.end err out.

****************************************************


Executing service control script:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/adapcctl.sh start
script returned:
****************************************************

You are running adapcctl.sh version 120.7.12010000.2

Starting OPMN managed Oracle HTTP Server (OHS) instance ...
opmnctl: opmn is already running.
opmnctl: starting opmn managed processes...

adapcctl.sh: exiting with status 0

adapcctl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_/logs/appl/admin/log/adapcctl.txt for more information ...


.end std out.

.end err out.

****************************************************


Executing service control script:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/adoacorectl.sh start

Executing service control script:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/adoacorectl.sh start
script returned:
****************************************************

You are running adoacorectl.sh version 120.13

Starting OPMN managed OACORE OC4J instance  ...

adoacorectl.sh: exiting with status 0

adoacorectl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_/logs/appl/admin/log/adoacorectl.txt for more information ...


.end std out.

.end err out.

****************************************************


Executing service control script:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/adformsctl.sh start
script returned:
****************************************************

You are running adformsctl.sh  version 120.16.12010000.3

Starting OPMN managed FORMS OC4J instance  ...
Calling txkChkFormsDeployment.pl to check whether latest FORMSAPP.EAR is deployed...
Program : /u01/E-BIZ/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkChkFormsDeployment.pl started @ Mon Nov  5 12:57:59 2012

*** Log File = /u01/E-BIZ/inst/apps/VISION_/logs/appl/rgf/TXK/txkChkFormsDeployment_Mon_Nov_5_12_57_59_2012/txkChkFormsDeployment_Mon_Nov_5_12_57_59_2012.log

File "/u01/E-BIZ/apps/tech_st/10.1.3/j2ee/forms/applications/forms/formsweb/WEB-INF/lib/frmsrv.jar" exists. Proceeding to check the size...

=============================================
*** Latest formsapp.ear has been deployed ***
=============================================


Program : /u01/E-BIZ/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkChkFormsDeployment.pl completed @ Mon Nov  5 12:58:00 2012

Perl script txkChkFormsDeployment.pl got executed successfully



adformsctl.sh: exiting with status 0

adformsctl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_/logs/appl/admin/log/adformsctl.txt for more information ...


.end std out.
*** ALL THE FOLLOWING FILES ARE REQUIRED FOR RESOLVING RUNTIME ERRORS
*** Log File = /u01/E-BIZ/inst/apps/VISION_i/logs/appl/rgf/TXK/txkChkFormsDeployment_Mon_Nov_5_12_57_59_2012/txkChkFormsDeployment_Mon_Nov_5_12_57_59_2012.log

.end err out.

****************************************************


Executing service control script:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/adoafmctl.sh start
script returned:
****************************************************

You are running adoafmctl.sh version 120.8

Starting OPMN managed OAFM OC4J instance  ...

adoafmctl.sh: exiting with status 0

adoafmctl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_/logs/appl/admin/log/adoafmctl.txt for more information ...


.end std out.

.end err out.

****************************************************


Executing service control script:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/adcmctl.sh start
script returned:
****************************************************

You are running adcmctl.sh version 120.17.12010000.5

Starting concurrent manager for VISION ...
Starting VISION_1105@VISION Internal Concurrent Manager
Default printer is noprint

adcmctl.sh: exiting with status 0


adcmctl.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_logs/appl/admin/log/adcmctl.txt for more information ...


.end std out.

.end err out.

****************************************************


Executing service control script:
/u01/E-BIZ/inst/apps/VISION_/admin/scripts/jtffmctl.sh start
script returned:
****************************************************

You are running jtffmctl.sh version 120.3

Validating Fulfillment patch level via /u01/E-BIZ/apps/apps_st/comn/java/classes
Fulfillment patch level validated.
Starting Fulfillment Server for VISION on port 9300 ...

jtffmctl.sh: exiting with status 0


.end std out.

.end err out.

****************************************************


All enabled services for this node are started.

adstrtal.sh: Exiting with status 0

adstrtal.sh: check the logfile /u01/E-BIZ/inst/apps/VISION_/logs/appl/admin/log/adstrtal.log for more information ...

You have new mail in /var/spool/mail/oracle
[oracle@ bin]$

Edited to remove all identifying host information. 







Friday, November 2, 2012

Configuring the E-Business DB Tier on Amazon

Relevant ML note is 1205963.1

Our variables we will use :  

  1. Apps password : must be apps or nothing works !!!  
  2. System hostname : this will be the internal hostname.  usually starts with ip-xxx=xxx=xxx=xxx.ec2.internal 
  3. Target is RAC : No 
  4. Server Node > same as hostname
  5. SID : VISION
  6. Target Base : /u01/E-BIZ/db
  7. utl_file_dir: /usr/tmp 
  8. DATA_TOPs  1
  9. DATA_TOP Location /u01/E-BIZ/db/apps_st/data
  10. RDBMS OH /u01/E-BIZ/db/tech_st/11.2.0.2
  11. preserve display : n
  12. Target display : default 
  13. Same Ports : n
  14. Port Pool : 0 
(All above as per manual )



Log on and su to oracle.

cd /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin

Execute the DB tier post-clone script




perl adcfgclone.pl dbTier


                     Copyright (c) 2002 Oracle Corporation

                    Copyright (c) 2002 Oracle Corporation
                        Redwood Shores, California, USA

                        Oracle Applications Rapid Clone

                                 Version 12.0.0

                      adcfgclone Version 120.31.12010000.8

Enter the APPS password :

Running:
/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/../jre/bin/java -Xmx600M -cp /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/java:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/xmlparserv2.jar:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/ojdbc5.jar oracle.apps.ad.context.CloneContext -e /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/../context/db/CTXORIG.xml -validate -pairsfile /tmp/adpairsfile_20475.lst -stage /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone  2> /tmp/adcfgclone_20475.err; echo $? > /tmp/adcfgclone_20475.res

Log file located at /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/CloneContext_1102114625.log

Provide the values required for creation of the new Database Context file.

Target System Hostname (virtual or normal) [ip-foo] :

Target Instance is RAC (y/n) [n] :
ERROR: Context creation not completed successfully.
For additional details review the file /tmp/adcfgclone_20475.err if present.

[oracle bin]$ perl adcfgclone.pl dbTier

                     Copyright (c) 2002 Oracle Corporation
                        Redwood Shores, California, USA

                        Oracle Applications Rapid Clone

                                 Version 12.0.0

                      adcfgclone Version 120.31.12010000.8

Enter the APPS password :

Running:
/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/../jre/bin/java -Xmx600M -cp /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/java:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/xmlparserv2.jar:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/ojdbc5.jar oracle.apps.ad.context.CloneContext -e /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/../context/db/CTXORIG.xml -validate -pairsfile /tmp/adpairsfile_20553.lst -stage /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone  2> /tmp/adcfgclone_20553.err; echo $? > /tmp/adcfgclone_20553.res

Log file located at /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/CloneContext_1102114637.log

Provide the values required for creation of the new Database Context file.

Target System Hostname (virtual or normal) [ip-] :

Target Instance is RAC (y/n) [n] :

Target System Database SID : VISION

Target System Base Directory : /u01/E-BIZ/db

Target System utl_file_dir Directory List : /usr/tmp

Number of DATA_TOP's on the Target System [1] :

Target System DATA_TOP Directory 1 [/u01/E-BIZ/db/apps_st/data] : /u01/E-BIZ/db/apps_st/data

Target System RDBMS ORACLE_HOME Directory [/u01/E-BIZ/db/db/tech_st/11.1.0] : /u01/E-BIZ/db/tech_st/11.2.0.2

Do you want to preserve the Display [atgtxk05:0.0] (y/n)  : n

Target System Display [ip-:0.0] :

Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n

Target System Port Pool [0-99] : 0

Checking the port pool 0
done: Port Pool 0 is free
Report file located at /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/temp/portpool.lst
Complete port information available at /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/temp/portpool.lst

Creating the new Database Context file from :
  /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/template/adxdbctx.tmp

The new database context file has been created :
  /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/VISION_.xml

Log file located at /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/CloneContext_1102114637.log
Check Clone Context logfile /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/CloneContext_1102114637.log for details.

Running Rapid Clone with command:
perl /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/adclone.pl java=/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/../jre mode=apply stage=/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone component=dbTier method=CUSTOM dbctxtg=/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/VISION_.xml showProgress contextValidated=true
Running:
perl /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/adclone.pl java=/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/../jre mode=apply stage=/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone component=dbTier method=CUSTOM dbctxtg=/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/VISION_.xml showProgress contextValidated=true
APPS Password :

Beginning database tier Apply - Fri Nov  2 11:50:13 2012

/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/bin/../jre/bin/java -Xmx600M -DCONTEXT_VALIDATED=true  -Doracle.installer.oui_loc=/u01/E-BIZ/db/tech_st/11.2.0.2/oui -classpath /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/xmlparserv2.jar:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/ojdbc5.jar:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/java:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/oui/OraInstaller.jar:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/oui/ewt3.jar:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/oui/share.jar:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/oui/srvm.jar:/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone/jlib/ojmisc.jar   oracle.apps.ad.clone.ApplyDBTier -e /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/VISION_.xml -stage /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/clone   -showProgress
APPS Password : Log file located at /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/log/VISION_-30/ApplyDBTier_11021150.log
  |      0% completed


Finally....

Log file located at /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/log/VISION_/ApplyDBTier_11021348.log
  /     15% completed

Completed Apply...
Fri Nov  2 13:53:29 2012

Starting database listener for VISION:
Running:
/u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/scripts/VISION_/addlnctl.sh start VISION
Logfile: /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/log/VISION_/addlnctl.txt

You are running addlnctl.sh version 120.1.12010000.4


Starting listener process VISION ...


Listener VISION has already been started.


addlnctl.sh: exiting with status 0

addlnctl.sh: check the logfile /u01/E-BIZ/db/tech_st/11.2.0.2/appsutil/log/VISION_/addlnctl.txt for more information ...


Had to repeat twice as it wasn't clear that the precloned db was a locked apps password.  However all OK now we will fix the apps password at the very end !




Removing the Hostname from the linux prompt.

So you don't want to have the hostname showing up in blog pastes, or it's just too long for the screen.

First thought was /etc/profile

It's not there !

But look at  /etc/bashrc


if [ "$PS1" ]; then
  if [ -z "$PROMPT_COMMAND" ]; then
    case $TERM in
    xterm*)
        if [ -e /etc/sysconfig/bash-prompt-xterm ]; then
            PROMPT_COMMAND=/etc/sysconfig/bash-prompt-xterm
        else
            PROMPT_COMMAND='printf "\033]0;%s@%s:%s\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/~}"'
        fi
        ;;
    screen)
        if [ -e /etc/sysconfig/bash-prompt-screen ]; then
            PROMPT_COMMAND=/etc/sysconfig/bash-prompt-screen
        else
            PROMPT_COMMAND='printf "\033]0;%s@%s:%s\033\\" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/~}"'
        fi
        ;;
    *)
        [ -e /etc/sysconfig/bash-prompt-default ] && PROMPT_COMMAND=/etc/sysconfig/bash-prompt-default
        ;;
    esac
  fi
  # Turn on checkwinsize
  shopt -s checkwinsize
  [ "$PS1" = "\\s-\\v\\\$ " ] && PS1="[\u@\h \W]\\$ "
fi


99% of the above doesn't need to be changed.  Just remove the last @\h  in the second last line and all should be well. 

Sorting out the Mount Points properly for our new EBS tier

Next thing to do is to dismount /u01 and replace it with /u02 and vice versa.

umount /u01
umount /u02

now edit fstab

LABEL=ebs1211db         /u01                    ext3    defaults        1 2
/dev/md0                 /u02                    ext3    defaults       0 0

just change the mount points round

LABEL=ebs1211db         /u02                    ext3    defaults        1 2
/dev/md0                 /u01                    ext3    defaults       0 0




[root@]# mount /u01
[root@]# mount /u02
[root@]# mount
/dev/sda2 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/md0 on /u01 type ext3 (rw)
/dev/sdb1 on /u02 type ext3 (rw)


So that's all OK 

Thursday, November 1, 2012

Next steps ... build a stable filesystem and copy over the database files etc.

OK so the next step then is once that mdstat is built

Personalities : [raid10] [raid6] [raid5] [raid4]
md0 : active raid10 sdo[9] sdn[8] sdm[7] sdl[6] sdk[5] sdj[4] sdi[3] sdh[2] sdg[1] sdf[0]
      314571520 blocks 256K chunks 2 near-copies [10/10] [UUUUUUUUUU]

unused devices:

Next step is to build a filesystem on it 

mkfs.ext3 /dev/md0 

Once that is done we will temporarily call it /u02 and now we will copy over the data 
add 
/dev/md0                 /u02                    ext3    defaults       0 0
to fstab
mount /u02 

We don' want to sit and watch the copy from the u01 

  nohup cp -a -v * /u02>cp1.log 2>&1 &

so now we can log out and go off to bed. We will check in again tomorrow  




Building a safe place for our E-Business Amazon EC2 database

As part of our Amazon build we need a huge amount of reasonably fast and safe space for the database.
We do this via construction of a RAID 10 volume of 10 EBS 64M volumes.

This will take some time so let's get it going immediately.


Strapping together an mdadm volume for RAID


root ~]# yum install mdadm
el5_u3_base                                                                                                                                                           | 1.1 kB     00:00
el5_u1_base                                                                                                                                                           | 1.1 kB     00:00
el5_ga_base                                                                                                                                                           | 1.1 kB     00:00
ol5_u5_base                                                                                                                                                           |  951 B     00:00
el5_u5_base                                                                                                                                                           | 1.1 kB     00:00
el5_u2_base                                                                                                                                                           | 1.1 kB     00:00
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
--> Running transaction check
---> Package mdadm.x86_64 0:2.6.9-3.el5 set to be updated
--> Finished Dependency Resolution

Install      1 Package(s)
Update       0 Package(s)
Remove       0 Package(s)

Total download size: 876 k
Is this ok [y/N]: y
Downloading Packages:
mdadm-2.6.9-3.el5.x86_64.rpm                                                                                                                                          | 876 kB     00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : mdadm                                             [1/1]

Installed: mdadm.x86_64 0:2.6.9-3.el5
Complete!
[root@ ~]#

now we being creating a raid10 array with 10 devices with 256k chunks


 mdadm --create /dev/md0 --level=10 --raid-devices=10 /dev/sd[fghijklmno]

make sure the create has begun by checking mdstat 


[root@~]#  cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 sdo[9] sdn[8] sdm[7] sdl[6] sdk[5] sdj[4] sdi[3] sdh[2] sdg[1] sdf[0]
      314571520 blocks 256K chunks 2 near-copies [10/10] [UUUUUUUUUU]
      [>....................]  resync =  2.2% (7133112/314571520) finish=95.2min speed=53807K/sec


.... Call back in 95.2 minutes when your raid build should be complete.