I encountered an issue while using CDC in Cassandra and need your help to resolve it.
I have set up a Change Data Capture (CDC) system to track changes in a specific table. CDC is enabled, and I can see that binary logs are being created in the cdc_raw directory. However, I noticed that the data modify (insert new records, update existing ones, or delete records) in this table does not appear in the cdc_raw logs. That is, the logs remain empty.
For reading CDC binary logs (cdc_raw), I rely on a pre-built .jar file that implements the Java class CommitLogReader.java
Below is my sequence of steps:
1 Installed Cassandra 4.1.8 in Docker
2 Set the variables in the file /etc/cassandra/cassandra.yaml
cdc_enabled: true
cdc_raw_directory: /var/lib/cassandra/cdc_raw
commitlog_directory: /var/lib/cassandra/commitlog
cdc_total_space: 4096MiB
cdc_free_space_check_interval: 250ms
3 Created a keyspace demo
CREATE KEYSPACE demo WITH REPLICATION = { 'class' : 'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': '3' };
4 Created a table demo.my_table_cdc_text
CREATE TABLE demo.my_table_cdc_text(
id int,
code text,
first_name text,
last_name text,
email text,
PRIMARY KEY(id, code)
)
WITH cdc=true;
5 Added data to the table demo.my_table_cdc_text
(350 rows)
BEGIN batch
INSERT INTO demo.my_table_cdc_text (id, code, email, first_name, last_name) VALUES(769, 'brown fox jumps over t', 'Joelle.Stuart.11@yahoo.in', 'Abraham', 'Douglas');
INSERT INTO demo.my_table_cdc_text (id, code, email, first_name, last_name) VALUES(769, 'dogThe quick brown fox jumps ove', 'Calvin.Haines.02@hetnet.nl', 'Oscar', 'Jeremy');
INSERT INTO demo.my_table_cdc_text (id, code, email, first_name, last_name) VALUES(769, 'dogThe quick brown fox jumps over t', 'Mercedes.Pippen.46@yahoo.es', 'Maggie', 'Roberta');
.........
INSERT INTO demo.my_table_cdc_text (id, code, email, first_name, last_name) VALUES(769, 'fox jumps over', 'Teresa.Plumb.84@laposte.net', 'Ida', 'Tamara');
INSERT INTO demo.my_table_cdc_text (id, code, email, first_name, last_name) VALUES(769, 'fox jumps over the lazy dogThe quick bro', 'Hodierna.Nicol.73@terra.com.br', 'Rich', 'Beatrice');
apply batch;
P.S: I also tried these parameters and commands:
- changed the
log_segment_size parameter
from32MiB
to1Mib
; - run command:
nodetool flush demo my_table_cdc_text