Scripted RPD deployments using UDML and WLST

Automating the RPD deployment in OBIEE 10g was as simple as doing an SCP of the RPD to the OBIEE server, and then bounce the processes. With OBIEE 11g, RPD deployments are now handled through Weblogic, so we need to take some additional steps. Luckily these can be easily scripted thanks to WLST.

After the jump is a script which can be used to do OBIEE 11g RPD deployments. This script assumes you have setup ssh keys on the various servers so you can login without typing a password (http://www.linuxproblem.org/art_9.html).


At a high level, the script will:

• Apply a custom UDML file to your RPD.
• See previous post on how/why to create UDML.
• Remotely connect to the destination, and SCP the RPD file.
• Clean up the destination repository directory to archive all but the last three RPD files
• This is important, as each deployment creates a new RPD (numbering from 001, 002, 003, etc). The directory can fill up quickly with lots of deployments.
• Call WLST with a custom python script to:
• Perform a lock
• Deploy the RPD
• Commit the changes
• Restart the BI server

We had an additional requirement to change the RPD password in an automated fashion for our QA server. So that is what is going on in the if [ "$1" = "QA" ] section of code. Right now, the new password is hardcoded in the script, but this can easily be changed to be an optional parameter of the script itself.

Click me to view deployRPD.sh

#!/bin/bash

######
# deployRPD.sh
# Created by Ben Mackin (bmackin@mac.com)
#
# This script will deploy an RPD
######

set -o pipefail

checkStatus ()
{
   if [ $1 -ne 0 ]; then

      if [ "${3-zz}" != "zz" ]; then
         exec $3
      fi

      echo -e "\n`date +[%H:%M:%S]` ###### FAILED ######\n\n" | tee -a $LOGFILE
      echo "The deployment failed: $2" | mutt -a $LOGFILE -s "$SUBJECT - FAILED" $EMAIL
      exit 1
   fi

   logOutput "Success\n"
   return 0
}

logOutput ()
{
   echo -e "`date +[%H:%M:%S]` - ${SERVERHOST} ## $1" | tee -a $LOGFILE
   return 0
}

# If there are less than 2 parameters, then show help and exit
if [ $# -lt 4 ]; then
   echo -e "\nUsage: deployRPD.sh -h host -u user [-n -a]\n\t-h\tWeblogic host you are migrating to.\n\t-u\tUsername of admin Weblogic user.\n\t-n\tTurn off logging at the RPD level\n\t-a\tChange the RPD password to match the destination weblogic password.\n"
   exit -1
fi

set -o pipefail

while getopts "h:u:na" opt; do
  case $opt in
    h) SERVERHOST=${OPTARG} ;;
    u) WEBLOGIC_USER=${OPTARG} ;;
    n) NOLOG=1 ;;
    a) CHANGERPDPASS=1 ;;
    \?) echo "Invalid option: -$OPTARG" | tee -a $LOGFILE; exit ;;
    :) echo "Option -$OPTARG requires an argument." | tee -a $LOGFILE; exit ;;
  esac
done

# We stored the weblogic password on the destination server for some other scripts, so just go grab it.
# Otherwise, you will want to replace this with your weblogic password
WEBLOGIC_PASS=`ssh ${SERVERHOST} 'tail -n 1 ${FMWHOME}/user_projects/domains/bifoundation_domain/servers/AdminServer/security/custom.sec.properties'`
ADMINPWD=Admin123 # change this to the admin password of the 11g RPD file

SUPPORT_LOC=~/support # folder for support scripts
SVN_LOC=~/svnget # change this to the path of where the RPD to deploy is
UDML_LOC=~/udml/${SERVERHOST}.udml # change this to the path of the udml scripts
RPD_NAME=EnterpriseBusinessAnalytics7963.rpd # change this to the name of the RPD file
LOGFILE=~/logs/${SERVERHOST}_rpddeploy_$(date +%y%m%d_%H%M%S).log
FMWHOME=/apps/applobi/OBIEE11g # change this to the fusion middleware home
DEPLOY_DIR=${FMWHOME}/instances/instance1/bifoundation/OracleBIServerComponent/coreapplication_obis1/repository

SUBJECT="OBIEE RPD Deployment on $SERVERHOST - `date +%m/%d/%Y` @ `date +%H:%M:%S`"
EMAIL=null@null.com

logOutput "Doing RPD deployment to $SERVERHOST\n"

##
# Apply the appropriate UDML to the RPD from SVN
##

# Check for lock file. Two nqudmlexec can't process the same .rpd, or the command locks up.
# If there is a lock file, wait 2 seconds and check again. Otherwise, process away!
while [ -f "$SVN_LOC/lockfile" ]
do
   sleep `expr $RANDOM % 10`
done

touch $SVN_LOC/lockfile

# Source the bi-init which sets up the environment variables for OBIEE commands
. ${FMWHOME}/instances/instance1/bifoundation/OracleBIApplication/coreapplication/setup/bi-init.sh

logOutput "Apply the UDML to the file from SVN\n"

# Apply a custom UDML which set the LOGLEVEL variable to 0 to disable logging
if [ ${NOLOG-0} -eq 1 ]; then
   logOutput "Apply the NOLOG option to the UDML and RPD\n"
   cat $UDML_LOC ~/bmackin/udml/NOLOG.udml > /tmp/${SERVERHOST}_COMBINE.udml
   ${FMWHOME}/Oracle_BI1/bifoundation/server/bin/nqudmlexec -P $ADMINPWD -I /tmp/${SERVERHOST}_COMBINE.udml -B $SVN_LOC/$RPD_NAME -O $SVN_LOC/${SERVERHOST}_$RPD_NAME.new 2>&1 | tee -a $LOGFILE
   checkStatus "$?" "The deployment failed: nqudmlexec.sh -P $ADMINPWD -I /tmp/${SERVERHOST}_COMBINE.udml -B $SVN_LOC/$RPD_NAME -O $SVN_LOC/${SERVERHOST}_$RPD_NAME.new" "rm $SVN_LOC/lockfile"
   rm /tmp/${SERVERHOST}_COMBINE.udml
else
   ${FMWHOME}/Oracle_BI1/bifoundation/server/bin/nqudmlexec -P $ADMINPWD -I $UDML_LOC -B $SVN_LOC/$RPD_NAME -O $SVN_LOC/${SERVERHOST}_$RPD_NAME.new 2>&1 | tee -a $LOGFILE
   checkStatus "$?" "The deployment failed: nqudmlexec.sh -P $ADMINPWD -I $UDML_LOC -B $SVN_LOC/$RPD_NAME -O $SVN_LOC/${SERVERHOST}_$RPD_NAME.new" "rm $SVN_LOC/lockfile"
fi

# Cleanup the temp files which the nqudmlexec command generates
rm $SVN_LOC/lockfile
rm $SVN_LOC/*.Log
rm -rf ${FMWHOME}/instances/instance1/bifoundation/OracleBIServerComponent/$SERVERHOST

# Check if the change password parameter was set
if [ ${CHANGERPDPASS-0} -eq 1 ]; then
   OLDADMINPWD=$ADMINPWD
   ADMINPWD=$WEBLOGIC_PASS # We are just setting the RPD password to the weblogic password
   logOutput "Change the RPD password for $SERVERHOST.\n"
   echo -e "${OLDADMINPWD}\n${ADMINPWD}" | ${FMWHOME}/Oracle_BI1/bifoundation/server/bin/obieerpdpwdchg -I $SVN_LOC/${SERVERHOST}_$RPD_NAME.new -O $SVN_LOC/${SERVERHOST}_$RPD_NAME.new2
   checkStatus "$?" "The deployment failed: obieerpdpwdchg -I $SVN_LOC/${SERVERHOST}_$RPD_NAME.new -O $SVN_LOC/${SERVERHOST}_$RPD_NAME.new2"

   rm $SVN_LOC/*.Log
   rm $SVN_LOC/${SERVERHOST}_$RPD_NAME.new
   mv $SVN_LOC/${SERVERHOST}_$RPD_NAME.new2 $SVN_LOC/${SERVERHOST}_$RPD_NAME.new
fi

#SSH to remote machine, clean up old files, deploy the newly gotten file.
logOutput "Clear out the .Log and .Sav files on $SERVERHOST\n"
ssh $SERVERHOST "rm '$DEPLOY_DIR'/*.Log" 2>&1 | tee -a $LOGFILE
ssh $SERVERHOST "rm '$DEPLOY_DIR'/*.Sav" 2>&1 | tee -a $LOGFILE
ssh $SERVERHOST "mkdir $DEPLOY_DIR/archive" 2>&1 | tee -a $LOGFILE
logOutput "Success\n"

# Each RPD deployment creates a new numbered file. Move all but the last 3 to the archive folder
logOutput "Save only the last 3 rpd file, move everything else to the archive folder\n"
ssh $SERVERHOST 'bash -s' < $SUPPORT_LOC/cleanuprpd.sh 2>&1 | tee -a $LOGFILE
logOutput "Success\n"

# Move the RPD to the deployment directory
logOutput "Move the file to the deployment directory\n"
scp $SVN_LOC/${SERVERHOST}_$RPD_NAME.new $SERVERHOST:$DEPLOY_DIR/$RPD_NAME 2>&1 | tee -a $LOGFILE
checkStatus "$?" "The deployment failed: scp $SVN_LOC/${SERVERHOST}_$RPD_NAME.new $SERVERHOST:$DEPLOY_DIR/$RPD_NAME"

# Cleanup the temp files
rm $SVN_LOC/${SERVERHOST}_$RPD_NAME.new

# Run WLST with the custom deployment script
logOutput "Do weblogic scripting to deploy the file\n"
${FMWHOME}/Oracle_BI1/common/bin/wlst.sh $SUPPORT_LOC/wlstDeployRPD.py -u $WEBLOGIC_USER -p $WEBLOGIC_PASS -h $SERVERHOST -o 7001 -r $ADMINPWD | tee -a $LOGFILE
checkStatus "$?" "The deployment failed: wlst.sh $SUPPORT_LOC/wlstDeployRPD.py -u $WEBLOGIC_USER -p $WEBLOGIC_PASS -h $SERVERHOST -o 7001 -r $ADMINPWD"

# Remove the pre-deployed file from the remove server
logOutput "Remove the pre-deployed file\n"
ssh $SERVERHOST "rm '$DEPLOY_DIR'/'$RPD_NAME'" 2>&1 | tee -a $LOGFILE
logOutput "Success\n"

# Clear the OBIEE cache on the remove server
logOutput "Clear the remote OBIEE cache\n"
ssh $SERVERHOST "cd $SUPPORT_LOC/..; ./clearCache.sh" 2>&1 | tee -a $LOGFILE

echo -e "\n`date +[%H:%M:%S]` - ${SERVERHOST} ## Deployment to $SERVERHOST was successfull." > ${SERVERHOST}_obi_status.tmp
ssh $SERVERHOST '${FMWHOME}/instances/instance1/bin/opmnctl status' >> ${SERVERHOST}_obi_status.tmp

cat ${SERVERHOST}_obi_status.tmp | tee -a $LOGFILE
cat ${SERVERHOST}_obi_status.tmp | mutt -a $LOGFILE -s "$SUBJECT - SUCCESSFULL" $EMAIL

rm ${SERVERHOST}_obi_status.tmp

Click me to view the wlstDeployRPD.py file

import sys
import os
from java.lang import System
import getopt

def usage():
    print "Usage:"
    print "genWlstDeployRPD -u user -p password -h host -o port -r adminpwd"

try:
    opts, args    = getopt.getopt(sys.argv[1:], "u:p:h:o:r:",
                                  ["user=", "password=", "host=", "port=", "adminpwd="])
except getopt.GetoptError, err:
    print str(err)
    usage()
    sys.exit(2)

user        = ''
password    = ''
host        = ''
port        = ''
adminpwd    = ''

for opt, arg in opts:
    if opt == "-u":
        user      = arg
    elif opt == "-p":
        password  = arg
    elif opt == "-h":
        host      = arg
    elif opt == "-o":
        port      = arg
    elif opt == "-r":
        adminpwd  = arg

if user == "":
    print "Missing \"-u user\" parameter."
    usage()
    sys.exit(2)
elif password == "":
    print "Missing \"-p password\" parameter."
    usage()
    sys.exit(2)
elif host == "":
    print "Missing \"-h host\" parameter."
    usage()
    sys.exit(2)
elif port == "":
    print "Missing \"-o port\" parameter."
    usage()
    sys.exit(2)
elif adminpwd == "":
    print "Missing \"-r adminpwd\" parameter."
    usage()
    sys.exit(2)

connect(user, password, host+":"+port)

# Be sure we are in the root
cd('..\..')

print(host + ': Connecting to Domain ...')
try:
    domainCustom()
except:
    print(host + ': Already in domainCustom')

print(host + ': Go to biee admin domain')
cd('oracle.biee.admin')

# go to the server configuration
print host + ': Go to BIDomain.BIInstance.ServerConfiguration MBean'
cd('oracle.biee.admin:type=BIDomain.BIInstance.ServerConfiguration,biInstance=coreapplication,group=Service')

# Lock the System
print(host + ': Calling lock ...')
cd('..')
cd('oracle.biee.admin:type=BIDomain,group=Service')
objs = jarray.array([], java.lang.Object)
strs = jarray.array([], java.lang.String)
try:
    invoke('lock', objs, strs)
except:
    print(host + ': System already locked')

cd('..')
# go to the server configuration
cd('oracle.biee.admin:type=BIDomain.BIInstance.ServerConfiguration,biInstance=coreapplication,group=Service')
print(host + ': Uploading RPD')
params =  jarray.array(['/apps/applobi/OBIEE11g/instances/instance1/bifoundation/OracleBIServerComponent/coreapplication_obis1/repository/AJ_EnterpriseBusinessAnalytics7963.rpd',adminpwd],java.lang.Object)
sign =  jarray.array(['java.lang.String', 'java.lang.String'],java.lang.String)
invoke( 'uploadRepository', params, sign)

# Commit the system
print(host + ': Commiting Changes')
cd('..')
cd('oracle.biee.admin:type=BIDomain,group=Service')
objs = jarray.array([], java.lang.Object)
strs = jarray.array([], java.lang.String)
try:
    invoke('commit', objs, strs)
except:
    print(host + ': System not locked')

# Restart the system
print(host + ': Restarting OBIEE processes')

cd('..\..')
cd('oracle.biee.admin')
cd('oracle.biee.admin:type=BIDomain.BIInstance,biInstance=coreapplication,group=Service')

print(host + ': Stopping the BI instance')
params = jarray.array([], java.lang.Object)
signs = jarray.array([], java.lang.String)
invoke('stop', params, signs)

BIServiceStatus = get('ServiceStatus')
print(host + ': BI ServiceStatus ' + BIServiceStatus)

print(host + ': Starting the BI instance')
params = jarray.array([], java.lang.Object)
signs = jarray.array([], java.lang.String)
invoke('start', params, signs)

BIServerStatus = get('ServiceStatus')
print(host + ': BI ServerStatus ' + BIServerStatus)