Illustration Image

12/9/2023

Reading time:6

Using Oracle GoldenGate for Big Data

Placeholder

This resource is based on an article originally published here.

DBOPTIONS ENABLECASSANDRACPPDRIVERTRACE true

Optional

Extract parameter (.prm) file.

false

Use only during initial load process.

When set to true, the Cassandra driver logs all the API calls to a driver.log file. This file is created in the Oracle GoldenGate for Big Data installation directory. This is useful for debugging.

DBOPTIONS FETCHBATCHSIZE number

Optional

Extract parameter (.prm) file.

1000

Minimum is 1

Maximum is 100000

Use only during initial load process.

Specifies the number of rows of data the driver attempts to fetch on each request submitted to the database server.

The parameter value should be lower than the database configuration parameter, tombstone_warn_threshold, in the database configuration file, cassandra.yaml. Otherwise the initial load process might fail.

Oracle recommends that you set this parameter value to 5000 for initial load Extract optimum performance.

TRANLOGOPTIONS CDCLOGDIRTEMPLATE path

Required

Extract parameter (.prm) file.

None

The CDC commit log directory path template. The template can optionally have the $nodeAddress meta field that is resolved to the respective node address.

TRANLOGOPTIONS SFTP options

Optional

Extract parameter (.prm) file.

None

The secure file transfer protocol (SFTP) connection details to pull and transfer the commit log files. You can use one or more of these options:

USER user

The SFTP user name.

PASSWORD password

The SFTP password.

KNOWNHOSTSFILE file

The location of the Secure Shell (SSH)known hosts file.

LANDINGDIR dir

The SFTP landing directory for the commit log files on the local machine.

PRIVATEKEY file

The SSH private key file.

PASSPHRASE password

The SSH private key pass phrase.

PORTNUMBER portnumber

The SSH port number.

CLUSTERCONTACTPOINTS nodes USER dbuser PASSWORD dbpassword

Optional

GLOBALS parameter file

127.0.0.1

A comma separated list of nodes to be used for a connection to the Cassandra cluster. You should provide at least one node address. The parameter options are:

USER dbuser

No default

Optional

The user name to use when connecting to the database.

PASSWORD dbpassword

No default

Required when USER is used.

The user password to use when connecting to the database.

PORT <port number

No default

Optional

The port to use when connecting to the database.

TRANLOGOPTIONS CDCREADERSDKVERSION version

Optional

Extract parameter (.prm) file.

3.11

The SDK Version for the CDC reader capture API.

ABENDONMISSEDRECORD | NOABENDONMISSEDRECORD

Optional

Extract parameter (.prm) file.

true

When set to true and the possibility of a missing record is found, the process stops with the diagnostic information. This is generally detected when a node goes down and the CDC reader doesn't find a replica node with a matching last record from the dead node. You can set this parameter to false to continue processing. A warning message is logged about the scenario.

TRANLOGOPTIONS CLEANUPCDCCOMMITLOGS

Optional

Extract parameter (.prm) file.

false

Purge CDC commit log files post extract processing. When the value is set to false, the CDC commit log files are moved to the cdc_raw_processed directory.

JVMBOOTOPTIONS jvm_options

Optional

GLOBALS parameter file

None

The boot options for the Java Virtual Machine. Multiple options are delimited by a space character.

JVMCLASSPATH classpath

Required

GLOBALS parameter file

None

The classpath for the Java Virtual Machine. You can include an asterisk (*) wildcard to match all JAR files in any directory. Multiple paths should be delimited with a colon (:) character.

OGGSOURCE source

Required

None

The source database for CDC capture or database queries. The valid value is CASSANDRA.

SOURCEDB nodeaddress USERID dbuser PASSWORD dbpassword

Required

Extract parameter (.prm) file.

None

A single Cassandra node address that is used for a connection to the Cassandra cluster and to query the metadata for the captured tables.

USER dbuser

No default

Optional

The user name to use when connecting to the database.

PASSWORD dbpassword

No default

Required when USER is used.

The user password to use when connecting to the database.

ABENDONUPDATERECORDWITHMISSINGKEYS | NOABENDONUPDATERECORDWITHMISSINGKEYS

Optional

Extract parameter (.prm) file.

true

If this value is true, anytime an UPDATE operation record with missing key columns is found, the process stops with the diagnostic information. You can set this property to false to continue processing and write this record to the trail file. A warning message is logged about the scenario. This operation is a partition update, see Partition Update or Insert of Static Columns.

ABENDONDELETERECORDWITHMISSINGKEYS | NOABENDONDELETERECORDWITHMISSINGKEYS

Optional

Extract parameter (.prm) file.

true

If this value is true, anytime an DELETE operation record with missing key columns is found, the process stops with the diagnostic information. You can set this property to false to continue processing and write this record to the trail file. A warning message is logged about the scenario. This operation is a partition update, see Partition Delete.

MOVECOMMITLOGSTOSTAGINGDIR | NOMOVECOMMITLOGSTOSTAGINGDIR

Optional

Extract parameter (.prm) file.

true

Enabled by default and this instructs the Extract group to move the commit log files in the cdc_raw directory on the Cassandra nodes to a staging directory for the commit log files. Only one Extract group can have movecommitlogstostagingdir enabled, and all the other Extract groups disable this by specifying nomovecommitlogstostagingdir.

SSL

Optional

GLOBALS or Extract parameter (.prm) file.

false

Use for basic SSL support during connection. Additional JSSE configuration through Java System properties is expected when enabling this.

CPPDRIVEROPTIONS SSL PEMPUBLICKEYFILE cassadra.pem

Optional

GLOBALS or Extract parameter (.prm) file.

String that indicates the absolute path with fully qualified name. This file is must for the SSL connection.

None, unless the PEMPUBLICKEYFILE property is specified, then you must specify a value.

Indicates that it is PEM formatted public key file used to verify the peer's certificate. This property is needed for one-way handshake or basic SSL connection.

CPPDRIVEROPTIONS SSL ENABLECLIENTAUTH | DISABLECLIENTAUTH

Optional

GLOBALS or Extract parameter (.prm) file.

false

Enabled indicates a two-way SSL encryption between client and server. It is required to authenticate both the client and the server through PEM formatted certificates. This property also needs the pemclientpublickeyfile and pemclientprivatekeyfile properties to be set. The pemclientprivatekeypasswd property must be configured if the client private key is password protected. Setting this property to false disables client authentication for two-way handshake.

CPPDRIVEROPTIONS SSL PEMCLIENTPUBLICKEYFILE public.pem

Optional

GLOBALS or Extract parameter (.prm) file.

String that indicates the absolute path with fully qualified name. This file is must for the SSL connection.

None, unless the PEMCLIENTPUBLICKEYFILE property is specified, then you must specify a value.

Use for a PEM formatted public key file name used to verify the client's certificate. This is must if you are using CPPDRIVEROPTIONS SSL ENABLECLIENTAUTH or for two-way handshake.

CPPDRIVEROPTIONS SSL PEMCLIENTPRIVATEKEYFILE public.pem

Optional

GLOBALS or Extract parameter (.prm) file.

String that indicates the absolute path with fully qualified name. This file is must for the SSL connection.

None, unless the PEMCLIENTPRIVATEKEYFILE property is specified, then you must specify a value.

Use for a PEM formatted private key file name used to verify the client's certificate. This is must if you are using CPPDRIVEROPTIONS SSL ENABLECLIENTAUTH or for two-way handshake.

CPPDRIVEROPTIONS SSL PEMCLIENTPRIVATEKEYPASSWD privateKeyPasswd

Optional

GLOBALS or Extract parameter (.prm) file.

A string

None, unless the PEMCLIENTPRIVATEKEYPASSWD property is specified, then you must specify a value.

Sets the password for the PEM formatted private key file used to verify the client's certificate. This is must if the private key file is protected with the password.

CPPDRIVEROPTIONS SSL PEERCERTVERIFICATIONFLAG value

Optional

GLOBALS or Extract parameter (.prm) file.

An integer

0

Sets the verification required on the peer's certificate. The range is 0–4:

0–Disable certificate identity verification.

1–Verify the peer certificate

2–Verify the peer identity

3– Not used so it is similar to disable certificate identity verification.

4 –Verify the peer identity by its domain name

CPPDRIVEROPTIONS SSL ENABLEREVERSEDNS

Optional

GLOBALS or Extract parameter (.prm) file.

false

Enables retrieving host name for IP addresses using reverse IP lookup.

Related Articles

logo
hive
elasticsearch
cassandra

Explore Further

etl

cassandra

spark

Become part of our
growing community!
Welcome to Planet Cassandra, a community for Apache Cassandra®! We're a passionate and dedicated group of users, developers, and enthusiasts who are working together to make Cassandra the best it can be. Whether you're just getting started with Cassandra or you're an experienced user, there's a place for you in our community.
A dinosaur
Planet Cassandra is a service for the Apache Cassandra® user community to share with each other. From tutorials and guides, to discussions and updates, we're here to help you get the most out of Cassandra. Connect with us and become part of our growing community today.
© 2009-2023 The Apache Software Foundation under the terms of the Apache License 2.0. Apache, the Apache feather logo, Apache Cassandra, Cassandra, and the Cassandra logo, are either registered trademarks or trademarks of The Apache Software Foundation. Sponsored by Anant Corporation and Datastax, and Developed by Anant Corporation.

Get Involved with Planet Cassandra!

We believe that the power of the Planet Cassandra community lies in the contributions of its members. Do you have content, articles, videos, or use cases you want to share with the world?