在Linux(AIX也适⽤)系统上安装OGG(⼀)
本⽂档主要介绍OGG在AIX系统上的安装,不适⽤其他操作系统。
整体架构
软件版本
OGG版本:
123010_ggs_Adapters_Linux_x64.zip (redhat)
122022_fbo_ggs_Linux_x64_shiphome.zip (redhat)
122022_fbo_ggs_AIX_ppc_shiphome.zip (AIX)
OGG for bigdata版本:
OGG_BigData_Linux_x64_12.3.2.1.1.zip
JDK版本:
jdk1.8
安装步骤
OGG安装(备库)
安装在/home/oracle⽬录中,使⽤oracle⽤户安装
1.上传安装包
将安装包122022_fbo_ggs_AIX_ppc_shiphome.zip上传⾄/home/oracle/temp/⽬录下,没有请⾃⾏创建
2.创建安装⽬录
mkdir /home/oracle/ogg
3.解压安装包
unzip 122022_fbo_ggs_AIX_ppc_shiphome.zip
cd fbo_ggs_AIX_ppc_shiphome/Disk1
4.静默安装
修改⽂件/home/oracle/temp/fbo_ggs_AIX_ppc_shiphome/Disk1/response/oggcore.rsp
修改以下位置:
INSTALL_OPTION=ORA12c
SOFTWARE_LOCATION=/home/oracle/ogg
START_MANAGER=false
MANAGER_PORT=7809
DATABASE_LOCATION=/u01/app/oracle/product/12.1.0/db_1
执⾏安装命令安装
./runInstaller -silent -responseFile /home/oracle/temp/fbo_ggs_AIX_ppc_shiphome/Disk1/response/oggcore.rsp Starting Oracle Universal Installer…
Checking Temp space: must be greater than 120 MB
. Actual 10461 MB Passed
Checking swap space: 0 MB available, 150 MB required. Failed <<<<
Some requirement checks failed. You must fulfill these requirements before
continuing with the installation,
Exiting Oracle Universal Installer, log for this session can be found at /tmp/OraInstall2019-11-07_10-41-
12AM/installActions2019-11-07_10-41-12AM.log
检查内存和swap
提⽰以下内容安装成功
The installation of Oracle GoldenGate Core was successful.
Please check ‘/u01/app/grid/logs/silentInstall2019-11-07_02-56-59PM.log’ for more details.
Successfully Setup Software.
OGG配置(备库)
1.添加环境变量
vi .profile
添加
export OGG_HOME=/home/oracle/ogg
export LIBPATH=:ORACLE_HOME/lib32:ORACLE_HOME/bin:GRID_HOME/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/java5/bin:OGG_HOME 退出保存
环境变量⽣效
. ~/.profile
2.主库操作
新建OGG⽤户及表空间,赋权限
(1)配置Oracle数据相关信息(主库执⾏)
开启数据库归档模式
linux安装数据库su - oracle
sqlplus / as sysdba (以DBA⾝份连接数据库)
SQL>alter database open; (打开数据库)
SQL>shutdown immediate; (⽴即关闭数据库)
SQL>startup mount; (启动实例并加载数据库,但不打开)
SQL>alter database archivelog; (更改数据库为归档模式)
SQL>alter database open; (打开数据库)
Oracle补全⽇志(Supplemental logging)特性因其作⽤的不同可分为以下⼏种:最⼩(Minimal),⽀持所有字段(all),⽀持主键(primary key),⽀持唯⼀键(unique),⽀持外键(foreign key)
开启附加⽇志(打开最⼩补全⽇志,只能库级别)
1)alter database add supplemental log data;
开启交换⽇志:
SQL> alter system switch logfile;
完成后,确保数据库处于归档模式,并已经开启附加⽇志和强制⽇志:
(2)创建tablespace和⽤户(主库执⾏)
创建⽤户并赋权:
备库操作
启⽤enable_goldengate_replication参数
alter system set enable_goldengate_replication = true;
3.OGG配置
(1)进⼊OGG_HOME,执⾏./ggsci登录,创建⽬录
GGSCI (coredg) 1> create subdirs
Creating subdirectories under current directory /home/oracle/ogg
Parameter files /home/oracle/ogg/dirprm: created
Report files /home/oracle/ogg/dirrpt: created
Checkpoint files /home/oracle/ogg/dirchk: created
Process status files /home/oracle/ogg/dirpcs: created
SQL script files /home/oracle/ogg/dirsql: created
Database definitions files /home/oracle/ogg/dirdef: created
Extract data files /home/oracle/ogg/dirdat: created
Temporary files /home/oracle/ogg/dirtmp: created
Credential store files /home/oracle/ogg/dircrd: created
Masterkey wallet files /home/oracle/ogg/dirwlt: created
Dump files /home/oracle/ogg/dirdmp: created
ORACLE OME /lib :H OGG OMEexportPATH =H ORACLE OME /OPatch :H PATH :
(2)编辑mgr进程
GGSCI (coredg) 5> edit params mgr
PORT 7809
DYNAMICPORTLIST 7810-7909
AUTORESTART EXTRACT ,RETRIES 5,WAITMINUTES 7
userid ogg@COREDB,password ogg
PURGEOLDEXTRACTS /home/oracle/ogg/dirdat/, usecheckpoints, minkeepdays 1
LAGREPORTHOURS 1
LAGINFOMINUTES 30
LAGCRITICALMINUTES 45
启动mgr进程
GGSCI (coredg) 7> start manager
Manager started.
查看进程状态
GGSCI (coredg) 8> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER STOPPED
查看进程⽇志
GGSCI (coredg) 10> view report mgr
(3)编辑抽取进程
GGSCI (coredg) 11> edit params ext0
EXTRACT ext0
setenv(ORACLE_HOME="/u01/app/oracle/product/12.1.0/db_1")
setenv(NLS_LANG=“AMERICAN_AMERICA.AL32UTF8”)
setenv(ORACLE_SID=“coredb”)
userid ogg@COREDB,password ogg
GETTRUNCATES
REPORTCOUNT EVERY 1 MINUTES, RATE
DISCARDFILE /home/oracle/ogg/dirrpt/ext0.dsc, APPEND, MEGABYTES 1000
WARNLONGTRANS 2h,CHECKINTERVAL 300
EXTTRAIL /home/oracle/ogg/dirdat/tr
TRANLOGOPTIONS EXCLUDEUSER ogg
TRANLOGOPTIONS MINEFROMACTIVEDG
DBOPTIONS ALLOWUNUSEDCOLUMN
DYNAMICRESOLUTION
REPORTROLLOVER AT 02:00
FETCHOPTIONS FETCHPKUPDATECOLS
GETUPDATEBEFORES
NOCOMPRESSDELETES
NOCOMPRESSUPDATES
IGNOREDELETES
IGNORETRUNCATES
TABLE ensemble.mb_tran_hist;
(4)在数据库中进⾏进程注册,增加抽取进程队列
2019-11-21 10:29:37 ERROR OGG-00868 The number of Oracle redo threads (1) is not the same as the number of checkpoint threads (2). EXTRACT groups on RAC systems should be created with the THREADS parameter (e.g., ADD EXT , TRANLOG, THREADS 1, BEGIN…
根据报错信息选择需要加⼊的threads
删除extract ext0
OGG登录
dblogin userid ogg@COREDB password ogg
delete extract ext0
GGSCI (coredg) 18> add extract ext0, tranlog, threads 1,begin now
EXTRACT added.
—从指定时间点开始抽取数据
查看抽取进程详细信息
GGSCI (coredg) > info ext9 detail
EXTRACT EXT9 Last Started 2019-12-03 15:10 Status RUNNING
Checkpoint Lag 00:00:00 (updated 00:00:08 ago)
Process ID 57511
Log Read Checkpoint Oracle Redo Logs
2019-12-03 17:06:13 Thread 1, Seqno 401, RBA 26073600
SCN 0.10769149 (10769149)
Log Read Checkpoint Oracle Redo Logs
2019-12-03 15:08:29 Thread 0, Seqno 0, RBA 0
SCN 0.0 (0)
Target Extract Trails:
Trail Name Seqno RBA Max MB Trail Type /home/oracle/ogg/dirdat/ext9/ex 2430 1615 500 EXTTRAIL Extract Source Begin End
Not Available 2019-12-03 15:08 2019-12-03 17:06
Not Available * Initialized * 2019-12-03 15:08
Not Available * Initialized * 2019-12-03 15:08
Not Available * Initialized * 2019-12-03 15:08
Not Available * Initialized * 2019-12-03 15:08
Not Available * Initialized * 2019-12-03 15:08
Current directory /home/oracle/ogg
Report file /home/oracle/ogg/dirrpt/EXT9.rpt
Parameter file /home/oracle/ogg/dirprm/ext9.prm
Checkpoint file /home/oracle/ogg/dirchk/EXT9.cpe
Process file /home/oracle/ogg/dirpcs/EXT9.pce
Error log /home/oracle/ogg/ggserr.log
查询SCN对应的时间点,从该时间点开始取数
add extract ext0, tranlog, threads 1,begin 2019-12-03 17:06:13
(5)最后添加trail⽂件的定义与extract进程绑定:
GGSCI (coredg) 20> ADD EXTTRAIL /home/oracle/ogg/dirdat/tr, EXTRACT ext0, megabytes 1 EXTTRAIL added.
(6)Start ext0启动抽取进程
GGSCI (coredg as ogg@coredb) 32> start ext0
查看进程状态info all
GGSCI (coredg as ogg@coredb) 35> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EXT0 00:00:00 00:02:44
(7)添加投递进程
GGSCI (coredg as ogg@coredb) 36> edit param pump0
extract pump0
dynamicresolution
gettruncates
userid ogg@COREDB,password ogg
RMTHOST 57.0.10.100, MGRPORT 7809
RMTFILE /home/oracle/oggbd/dirdat/tr, MEGABYTES 2, PURGE
TABLE ensemble.mb_tran_hist;
增加传送进程队列,分别将本地trail⽂件和⽬标端的trail⽂件绑定到extract进程:添加pump捕获组:
add extract pump0, exttrailsource /home/oracle/ogg/dirdat/tr,begin now 定义pump trail⽂件:
add rmttrail /home/oracle/oggbd/dirdat/tr, extract pump0 megabytes 1启动投递进程pump
GGSCI (coredg as ogg@coredb) 55> start pump0
GGSCI (coredg as ogg@coredb) 58> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EXT0 00:00:00 00:00:08
EXTRACT RUNNING PUMP0 00:00:00 00:00:07
OGG for Bigdata安装
Kafka会限制消息的⼤⼩,当超过限制时会舍弃该消息,需要修改消息⼤⼩限制参数:message.max.bytes
replica.fetch.max.bytes
1.切换到Oracle⽤户
su - oracle
2.创建java⽬录
mkdir -p /home/oracle/java
3.将软件包 jdk-8u191-linux-x6
以oracle⽤户上传到 /home/oracle/java⽬录
4.解压缩
tar -C /home/oracle/java
4.修改环境变量
vi .bash_profile
增加如下内容:
export OGG_HOME=/home/oracle/oggbd
export JAVA_HOME=/home/oracle/java/jdk1.8.0_191-amd64
export PATH=PATH
export LD_LIBRARY_PATH=JAVA_HOME/jre/lib/amd64:JAVA_HOME/jre/lib/amd64/libjsig.so:$OGG_HOME/lib
5.环境变量⽣效
source .profile
OGG for Bigdata配置
配置管理进程mgr
GGSCI (BigDataAPP) 10> edit param mgr JAV A OME /bin :H JAV A OME /jre /lib /amd 64/server :H JAV A OME /jre /lib /amd 64/server /libjvm .so :H